{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 第5章 卷积神经网络\n",
    "\n",
    "卷积神经网络（Convolutional Neural Network，CNN）是受生物学上感受野机制的启发而提出的。目前的卷积神经网络一般是由卷积层、汇聚层和全连接层交叉堆叠而成的前馈神经网络，有三个结构上的特性：局部连接、权重共享以及汇聚。这些特性使得卷积神经网络具有一定程度上的平移、缩放和旋转不变性。和前馈神经网络相比，卷积神经网络的参数更少。卷积神经网络主要应用在图像和视频分析的任务上，其准确率一般也远远超出了其他的神经网络模型。近年来卷积神经网络也广泛地应用到自然语言处理、推荐系统等领域。\n",
    "\n",
    "在学习本章内容前，建议您先阅读《神经网络与深度学习》第5章：卷积神经网络的相关内容，关键知识点如 **图5.1** 所示，以便更好的理解和掌握书中的理论知识在实践中的应用方法。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/4e6c4b60ff9b4de98db14a30c6ef153c8035f39ab8f7421189a4b96bc811ed7e\" width=500></center>\n",
    "<br><center>图5.1：《神经网络与深度学习》关键知识点回顾</center></br>\n",
    "\n",
    "本实践基于 **《神经网络与深度学习》第5章：卷积神经网络** 相关内容进行设计，主要包含两部分：\n",
    "\n",
    "* **模型解读**：介绍卷积的原理、卷积神经网络的网络结构、残差连接的原理以及残差网络的网络结构，并使用简单卷积神经网络和残差网络，完成手写数字识别任务；\n",
    "* **案例与实践**：基于残差网络ResNet18完成CIFAR-10图像分类任务。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 5.1 卷积\n",
    "\n",
    "考虑到使用全连接前馈网络来处理图像时，会出现如下问题：\n",
    "\n",
    "1. **模型参数过多，容易发生过拟合。** 在全连接前馈网络中，隐藏层的每个神经元都要跟该层所有输入的神经元相连接。随着隐藏层神经元数量的增多，参数的规模也会急剧增加，导致整个神经网络的训练效率非常低，也很容易发生过拟合。\n",
    "\n",
    "2. **难以提取图像中的局部不变性特征。** 自然图像中的物体都具有局部不变性特征，比如尺度缩放、平移、旋转等操作不影响其语义信息。而全连接前馈网络很难提取这些局部不变性特征。\n",
    "\n",
    "卷积神经网络有三个结构上的特性：局部连接、权重共享和汇聚。这些特性使得卷积神经网络具有一定程度上的平移、缩放和旋转不变性。和前馈神经网络相比，卷积神经网络的参数也更少。因此，通常会使用卷积神经网络来处理图像信息。\n",
    "\n",
    "**卷积**是分析数学中的一种重要运算，常用于信号处理或图像处理任务。本节以二维卷积为例来进行实践。\n",
    "\n",
    "### 5.1.1 二维卷积运算\n",
    "\n",
    "在机器学习和图像处理领域，卷积的主要功能是在一个图像（或特征图）上滑动一个卷积核，通过卷积操作得到一组新的特征。在计算卷积的过程中，需要进行卷积核的翻转，而这也会带来一些不必要的操作和开销。因此，在具体实现上，一般会以数学中的**互相关**（Cross-Correlatio）运算来代替卷积。\n",
    "在神经网络中，卷积运算的主要作用是抽取特征，卷积核是否进行翻转并不会影响其特征抽取的能力。特别是当卷积核是可学习的参数时，卷积和互相关在能力上是等价的。因此，很多时候，为方便起见，会直接用互相关来代替卷积。\n",
    "\n",
    "------\n",
    "**说明：**\n",
    "\n",
    "在本案例之后的描述中，除非特别声明，卷积一般指“互相关”。\n",
    "\n",
    "------\n",
    "\n",
    "对于一个输入矩阵$\\mathbf X\\in\\Bbb{R}^{M\\times N}$和一个滤波器$\\mathbf W \\in\\Bbb{R}^{U\\times V}$，它们的卷积为\n",
    "\n",
    "$$y_{i,j}=\\sum_{u=0}^{U-1} \\sum_{v=0}^{V-1} w_{uv}x_{i+u,j+v}。（5.1）$$\n",
    "\n",
    "------\n",
    "**说明：**\n",
    "\n",
    "这里和《神经网络与深度学习》中的定义区别是矩阵的下标从0开始。\n",
    "\n",
    "------\n",
    "\n",
    "**图5.2** 给出了卷积计算的示例。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/1b057e53beac41e8b721c34ba04df3e8e1420668e90c4793944e9da0eb18a962\" width = \"700\"></center>\n",
    "<center><br>图5.2：卷积操作的计算过程</br></center>\n",
    "\n",
    "经过卷积运算后，最终输出矩阵大小则为\n",
    "\n",
    "$$M' = M - U + 1,（5.2）$$\n",
    "$$N' = N - V + 1.（5.3）$$\n",
    "\n",
    "可以发现，使用卷积处理图像，会有以下两个特性：\n",
    "1. 在卷积层(假设是第$l$层)中的每一个神经元都只和前一层(第$l-1$层)中某个局部窗口内的神经元相连，构成一个局部连接网络，这也就是卷积神经网络的**局部连接**特性。\n",
    "2. 由于卷积的主要功能是在一个图像（或特征图）上滑动一个卷积核，所以作为参数的卷积核$\\mathbf W \\in\\Bbb{R}^{U\\times V}$对于第$l$层的所有的神经元都是相同的，这也就是卷积神经网络的**权重共享**特性。\n",
    "\n",
    "### 5.1.2 二维卷积算子\n",
    "\n",
    "********\n",
    "\n",
    "在本书后面的实现中，算子都继承`paddle.nn.Layer`，并使用支持反向传播的飞桨API进行实现，这样我们就可以不用手工写`backword()`的代码实现。\n",
    "\n",
    "********\n",
    "\n",
    "根据公式（5.1），我们首先实现一个简单的二维卷积算子，代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. \n",
      "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
      "  if data.dtype == np.object:\n",
      "W0715 10:38:18.376557   150 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 10.1\n",
      "W0715 10:38:18.381553   150 device_context.cc:465] device: 0, cuDNN Version: 7.6.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input: Tensor(shape=[1, 3, 3], dtype=float32, place=CUDAPlace(0), stop_gradient=True,\n",
      "       [[[1., 2., 3.],\n",
      "         [4., 5., 6.],\n",
      "         [7., 8., 9.]]]), \n",
      "output: Tensor(shape=[1, 2, 2], dtype=float32, place=CUDAPlace(0), stop_gradient=False,\n",
      "       [[[25., 31.],\n",
      "         [43., 49.]]])\n"
     ]
    }
   ],
   "source": [
    "import paddle\n",
    "import paddle.nn as nn\n",
    "\n",
    "class Conv2D(nn.Layer):\n",
    "    def __init__(self, kernel_size, \n",
    "                    weight_attr=paddle.ParamAttr(initializer=nn.initializer.Assign(value=[[0., 1.],[2., 3.]]))):\n",
    "        super(Conv2D, self).__init__()\n",
    "        # 使用'paddle.create_parameter'创建卷积核\n",
    "        # 使用'paddle.ParamAttr'进行参数初始化\n",
    "        self.weight = paddle.create_parameter(shape=[kernel_size,kernel_size],\n",
    "                                                dtype='float32',\n",
    "                                                attr=weight_attr)\n",
    "    def forward(self, X):\n",
    "        \"\"\"\n",
    "        输入：\n",
    "            - X：输入矩阵，shape=[B, M, N]，B为样本数量\n",
    "        输出：\n",
    "            - output：输出矩阵\n",
    "        \"\"\"\n",
    "        u, v = self.weight.shape\n",
    "        output = paddle.zeros([X.shape[0], X.shape[1] - u + 1, X.shape[2] - v + 1])\n",
    "        for i in range(output.shape[1]):\n",
    "            for j in range(output.shape[2]):\n",
    "                output[:, i, j] = paddle.sum(X[:, i:i+u, j:j+v]*self.weight, axis=[1,2])\n",
    "        return output\n",
    "\n",
    "# 随机构造一个二维输入矩阵\n",
    "paddle.seed(100)\n",
    "inputs = paddle.to_tensor([[[1.,2.,3.],[4.,5.,6.],[7.,8.,9.]]])\n",
    "\n",
    "conv2d = Conv2D(kernel_size=2)\n",
    "outputs = conv2d(inputs)\n",
    "print(\"input: {}, \\noutput: {}\".format(inputs, outputs))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.1.3 二维卷积的参数量和计算量\n",
    "\n",
    "**参数量**\n",
    "\n",
    "由于二维卷积的运算方式为在一个图像（或特征图）上滑动一个卷积核，通过卷积操作得到一组新的特征。所以参数量仅仅与卷积核的尺寸有关，对于一个输入矩阵$\\mathbf X\\in\\Bbb{R}^{M\\times N}$和一个滤波器$\\mathbf W \\in\\Bbb{R}^{U\\times V}$，卷积核的参数量为$U\\times V$。\n",
    "\n",
    "假设有一幅大小为$32\\times 32$的图像，如果使用全连接前馈网络进行处理，即便第一个隐藏层神经元个数为1，此时该层的参数量也高达$1025$个，此时该层的计算过程如 **图5.3** 所示。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/18faf1c2dfbb48659d2ccb72cd16e917f0af57b4640142ed8f98f9ceaa8b5ae8\" width = \"800\"></center>\n",
    "<center><br>图5.3：使用全连接前馈网络处理图像数据的计算过程 </br></center>\n",
    "<br></br>\n",
    "\n",
    "可以想像，随着隐藏层神经元数量的变多以及层数的加深，使用全连接前馈网络处理图像数据时，参数量会急剧增加。\n",
    "\n",
    "如果使用卷积进行图像处理，当卷积核为$3\\times 3$时，参数量仅为$9$，相较于全连接前馈网络，参数量少了非常多。\n",
    "\n",
    "**计算量**\n",
    "\n",
    "在卷积神经网络中运算时，通常会统计网络总的乘加运算次数作为计算量（FLOPs，floating point of operations），来衡量整个网络的运算速度。对于单个二维卷积，计算量的统计方式为：\n",
    "\n",
    "$$FLOPs=M'\\times N'\\times U\\times V。（5.4）$$\n",
    "\n",
    "其中$M'\\times N'$表示输出特征图的尺寸，即输出特征图上每个点都要与卷积核$\\mathbf W \\in\\Bbb{R}^{U\\times V}$进行$U\\times V$次乘加运算。对于一幅大小为$32\\times 32$的图像，使用$3\\times 3$的卷积核进行运算可以得到以下的输出特征图尺寸：\n",
    "\n",
    "$$M' = M - U + 1 = 30$$\n",
    "$$N' = N - V + 1 = 30$$\n",
    "\n",
    "此时，计算量为：\n",
    "\n",
    "$$FLOPs=M'\\times N'\\times U\\times V=30\\times 30\\times 3\\times 3=8100$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.1.4 感受野\n",
    "\n",
    "输出特征图上每个点的数值，是由输入图片上大小为$U\\times V$的区域的元素与卷积核每个元素相乘再相加得到的，所以输入图像上$U\\times V$区域内每个元素数值的改变，都会影响输出点的像素值。我们将这个区域叫做输出特征图上对应点的感受野。感受野内每个元素数值的变动，都会影响输出点的数值变化。比如$3\\times3$卷积对应的感受野大小就是$3\\times3$，如 **图5.4** 所示。\n",
    "\n",
    "<br></br>\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/1021536721524f4d8f4c1aefa89693c4b0fd388f21a347b583d413b3ac41241b\" width = \"800\"></center>\n",
    "<center><br>图5.4：感受野为3×3的卷积 </br></center>\n",
    "<br></br>\n",
    "\n",
    "而当通过两层$3\\times3$的卷积之后，感受野的大小将会增加到$5\\times5$，如 **图5.5** 所示。\n",
    "\n",
    "<br></br>\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/ac14916db81e40a48a25ab894d7a95e33fa0eece71d44a55af7bffab462fb7a7\" width = \"800\"></center>\n",
    "<center><br>图5.5：感受野为5×5的卷积 </br></center>\n",
    "<br></br>\n",
    "\n",
    "因此，当增加卷积网络深度的同时，感受野将会增大，输出特征图中的一个像素点将会包含更多的图像语义信息。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.1.5 卷积的变种\n",
    "\n",
    "在卷积的标准定义基础上，还可以引入卷积核的滑动步长和零填充来增加卷积的多样性，从而更灵活地进行特征抽取。\n",
    "\n",
    "#### 5.1.5.1 步长（Stride）\n",
    "\n",
    "在卷积运算的过程中，有时会希望跳过一些位置来降低计算的开销，也可以把这一过程看作是对标准卷积运算输出的**下采样**。\n",
    "\n",
    "在计算卷积时，可以在所有维度上每间隔$S$个元素计算一次，$S$称为卷积运算的**步长**（Stride），也就是卷积核在滑动时的间隔。\n",
    "\n",
    "此时，对于一个输入矩阵$\\mathbf X\\in\\Bbb{R}^{M\\times N}$和一个滤波器$\\mathbf W \\in\\Bbb{R}^{U\\times V}$，它们的卷积为\n",
    "\n",
    "$$y_{i,j}=\\sum_{u=0}^{U-1} \\sum_{v=0}^{V-1} w_{uv}x_{i\\times S+u,j\\times S+v}，（5.5）$$\n",
    "\n",
    "在二维卷积运算中，当步长$S=2$时，计算过程如 **图5.6** 所示。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/67bdbc3a81e945f4a6e469a40db5d41ac7f5e47c801f4ec9b9f4950d3d908160\" width = \"800\"></center>\n",
    "<center><br>图5.6：步长为2的二维卷积计算过程 </br></center>\n",
    "<br></br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 5.1.5.2 零填充（Zero Padding）\n",
    "\n",
    "在卷积运算中，还可以对输入用零进行填充使得其尺寸变大。根据卷积的定义，如果不进行填充，当卷积核尺寸大于1时，输出特征会缩减。对输入进行零填充则可以对卷积核的宽度和输出的大小进行独立的控制。\n",
    "\n",
    "在二维卷积运算中，**零填充**（Zero Padding）是指在输入矩阵周围对称地补上$P$个$0$。**图5.7** 为使用零填充的示例。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/deae0e6e83ba48968b153a2cc549c5bc8fc86d78c224441fb2e5700f3fea76ac\" width = \"200\"></center>\n",
    "<center><br>图5.7：padding=1的零填充 </br></center>\n",
    "<br></br>\n",
    "\n",
    "对于一个输入矩阵$\\mathbf X\\in\\Bbb{R}^{M\\times N}$和一个滤波器$\\mathbf W \\in\\Bbb{R}^{U\\times V}$，，步长为$S$，对输入矩阵进行零填充，那么最终输出矩阵大小则为\n",
    "\n",
    "$$M' = \\frac{M + 2P - U}{S} + 1,（5.6）$$\n",
    "$$N' = \\frac{N + 2P - V}{S} + 1.（5.7）$$\n",
    "\n",
    "引入步长和零填充后的卷积，参数量和计算量的统计方式与之前一致，参数量与卷积核的尺寸有关，为：$U\\times V$，计算量与输出特征图和卷积核的尺寸有关，为：\n",
    "\n",
    "$$FLOPs=M'\\times N'\\times U\\times V=(\\frac{M + 2P - U}{S} + 1)\\times (\\frac{N + 2P - V}{S} + 1)\\times U\\times V。（5.8）$$\n",
    "\n",
    "一般常用的卷积有以下三类：\n",
    "\n",
    "1. **窄卷积**：步长$S=1$，两端不补零$P=0$，卷积后输出尺寸为：\n",
    "\n",
    "$$M' = M - U + 1,（5.9）$$\n",
    "$$N' = N - V + 1.（5.10）$$\n",
    "\n",
    "2. **宽卷积**：步长$S=1$，两端补零$P=U-1=V-1$，卷积后输出尺寸为：\n",
    "\n",
    "$$M' = M + U - 1,（5.11）$$\n",
    "$$N' = N + V - 1.（5.12）$$\n",
    "\n",
    "3. **等宽卷积**：步长$S=1$，两端补零$P=\\frac{(U-1)}{2}=\\frac{(V-1)}{2}$，卷积后输出尺寸为：\n",
    "\n",
    "$$M' = M,（5.13）$$\n",
    "$$N' = N.（5.14）$$\n",
    "\n",
    "通常情况下，在层数较深的卷积神经网络，比如：VGG、ResNet中，会使用等宽卷积保证输出特征图的大小不会随着层数的变深而快速缩减。例如：当卷积核的大小为$3\\times 3$时，会将步长设置为$S=1$，两端补零$P=1$，此时，卷积后的输出尺寸就可以保持不变。在本章后续的案例中，会使用ResNet进行实验。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.1.6 带步长和零填充的二维卷积算子\n",
    "\n",
    "引入步长和零填充后，二维卷积算子代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "When kernel_size=3, padding=1 stride=1, input's shape: [2, 8, 8], output's shape: [2, 8, 8]\n",
      "When kernel_size=3, padding=1 stride=2, input's shape: [2, 8, 8], output's shape: [2, 4, 4]\n"
     ]
    }
   ],
   "source": [
    "class Conv2D(nn.Layer):\n",
    "    def __init__(self, kernel_size, stride=1, padding=0, \n",
    "                    weight_attr=paddle.ParamAttr(initializer=nn.initializer.Constant(value=1.0))):\n",
    "        super(Conv2D, self).__init__()\n",
    "        self.weight = paddle.create_parameter(shape=[kernel_size,kernel_size], \n",
    "                                                dtype='float32', \n",
    "                                                attr=weight_attr)\n",
    "        # 步长\n",
    "        self.stride = stride\n",
    "        # 零填充\n",
    "        self.padding = padding\n",
    "\n",
    "    def forward(self, X):\n",
    "        # 零填充\n",
    "        new_X = paddle.zeros([X.shape[0], X.shape[1]+2*self.padding, X.shape[2]+2*self.padding])\n",
    "        new_X[:, self.padding:X.shape[1]+self.padding, self.padding:X.shape[2]+self.padding] = X\n",
    "        u, v = self.weight.shape\n",
    "        output_w = (new_X.shape[1] - u) // self.stride + 1\n",
    "        output_h = (new_X.shape[2] - v) // self.stride + 1\n",
    "        output = paddle.zeros([X.shape[0], output_w, output_h])\n",
    "        for i in range(0, output.shape[1]):\n",
    "            for j in range(0, output.shape[2]):\n",
    "                output[:, i, j] = paddle.sum(\n",
    "                    new_X[:, self.stride*i:self.stride*i+u, self.stride*j:self.stride*j+v]*self.weight,\n",
    "                    axis=[1,2])\n",
    "        return output\n",
    "\n",
    "inputs = paddle.randn(shape=[2, 8, 8])\n",
    "conv2d_padding = Conv2D(kernel_size=3, padding=1)\n",
    "outputs = conv2d_padding(inputs)\n",
    "print(\"When kernel_size=3, padding=1 stride=1, input's shape: {}, output's shape: {}\".format(inputs.shape, outputs.shape))\n",
    "conv2d_stride = Conv2D(kernel_size=3, stride=2, padding=1)\n",
    "outputs = conv2d_stride(inputs)\n",
    "print(\"When kernel_size=3, padding=1 stride=2, input's shape: {}, output's shape: {}\".format(inputs.shape, outputs.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看出，使用$3\\times3$大小卷积，`padding`为1，当`stride`=1时，模型的输出特征图可以与输入特征图保持一致；当`stride`=2时，输出特征图的宽和高都缩小一倍。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.1.7 使用卷积运算完成图像边缘检测任务\n",
    "\n",
    "在图像处理任务中，常用**拉普拉斯算子**对物体边缘进行提取，拉普拉斯算子为一个大小为$3 \\times 3$的卷积核，中心元素值是$8$，其余元素值是$-1$。\n",
    "\n",
    "*******\n",
    "\n",
    "考虑到边缘其实就是图像上像素值变化很大的点的集合，因此可以通过计算二阶微分得到，当二阶微分为0时，像素值的变化最大。此时，对$x$方向和$y$方向分别求取二阶导数：\n",
    "\n",
    "$$\\frac{\\delta^2 I}{\\delta x^2} = I(i, j+1) - 2I(i,j) + I(i,j-1),（5.15）$$\n",
    "$$\\frac{\\delta^2 I}{\\delta y^2} = I(i+1, j) - 2I(i,j) + I(i-1,j).（5.16）$$\n",
    "\n",
    "完整的二阶微分公式为：\n",
    "\n",
    "$$\\nabla^2I = \\frac{\\delta^2 I}{\\delta x^2} + \\frac{\\delta^2 I}{\\delta y^2} =  - 4I(i,j) + I(i,j-1) + I(i, j+1) + I(i+1, j) + I(i-1,j),（5.17）$$\n",
    "\n",
    "上述公式也被称为**拉普拉斯算子**，对应的二阶微分卷积核为：\n",
    "\n",
    "$$\\begin{bmatrix}\n",
    "0      & 1   &  0    \\\\\n",
    "1      & -4 &  1 \\\\\n",
    "0      &  1 &  0 \\\\\n",
    "\\end{bmatrix}$$\n",
    "\n",
    "对上述算子全部求反也可以起到相同的作用，此时，该算子可以表示为：\n",
    "\n",
    "$$\\begin{bmatrix}\n",
    "0      & -1   &  0    \\\\\n",
    "-1      & 4 &  -1 \\\\\n",
    "0      &  -1 &  0 \\\\\n",
    "\\end{bmatrix}$$\n",
    "\n",
    "也就是一个点的四邻域拉普拉斯的算子计算结果是自己像素值的四倍减去上下左右的像素的和，将这个算子旋转$45°$后与原算子相加，就变成八邻域的拉普拉斯算子，也就是一个像素自己值的八倍减去周围一圈八个像素值的和，做为拉普拉斯计算结果，此时，该算子可以表示为：\n",
    "\n",
    "$$\\begin{bmatrix}\n",
    "-1      & -1   &  -1    \\\\\n",
    "-1      & 8 &  -1 \\\\\n",
    "-1      &  -1 &  -1 \\\\\n",
    "\\end{bmatrix}$$\n",
    "\n",
    "*******\n",
    "\n",
    "下面我们利用上面定义的`Conv2D`算子，构造一个简单的拉普拉斯算子，并对一张输入的灰度图片进行边缘检测，提取出目标的外形轮廓。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working\n",
      "  from collections import MutableMapping\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working\n",
      "  from collections import Iterable, Mapping\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working\n",
      "  from collections import Sized\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bf to_tensor, inputs: [[0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " ...\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]]\n",
      "bf unsqueeze, inputs: Tensor(shape=[256, 256], dtype=float32, place=CUDAPlace(0), stop_gradient=True,\n",
      "       [[0., 0., 0., ..., 0., 0., 0.],\n",
      "        [0., 0., 0., ..., 0., 0., 0.],\n",
      "        [0., 0., 0., ..., 0., 0., 0.],\n",
      "        ...,\n",
      "        [0., 0., 0., ..., 0., 0., 0.],\n",
      "        [0., 0., 0., ..., 0., 0., 0.],\n",
      "        [0., 0., 0., ..., 0., 0., 0.]])\n",
      "af unsqueeze, inputs: Tensor(shape=[1, 256, 256], dtype=float32, place=CUDAPlace(0), stop_gradient=True,\n",
      "       [[[0., 0., 0., ..., 0., 0., 0.],\n",
      "         [0., 0., 0., ..., 0., 0., 0.],\n",
      "         [0., 0., 0., ..., 0., 0., 0.],\n",
      "         ...,\n",
      "         [0., 0., 0., ..., 0., 0., 0.],\n",
      "         [0., 0., 0., ..., 0., 0., 0.],\n",
      "         [0., 0., 0., ..., 0., 0., 0.]]])\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/cbook/__init__.py:2349: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working\n",
      "  if isinstance(obj, collections.Iterator):\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/cbook/__init__.py:2366: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working\n",
      "  return list(data) if isinstance(data, collections.MappingView) else data\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/image.py:425: DeprecationWarning: np.asscalar(a) is deprecated since NumPy v1.16, use a.item() instead\n",
      "  a_min = np.asscalar(a_min.astype(scaled_dtype))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/image.py:426: DeprecationWarning: np.asscalar(a) is deprecated since NumPy v1.16, use a.item() instead\n",
      "  a_max = np.asscalar(a_max.astype(scaled_dtype))\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe0AAAD8CAYAAABaSfxxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsvXuQbNtZH/Zb/X6/5nXm3HPO1X3oIsEVBGRECKlwVQQFEhIZlyNB4cQyihVsyYkLAgalHCkoThQEwThQGIkCgSsIMIRAjAzm4WscG4JeqRJw9b6PM+cxMz393P1+rPyx+/fN6j27e7p7ume656xfVddMd+/H2t299m993/f7vk9prWFhYWFhYWGx/ghc9QAsLCwsLCwsZoMlbQsLCwsLiw2BJW0LCwsLC4sNgSVtCwsLCwuLDYElbQsLCwsLiw2BJW0LCwsLC4sNgSXtS4JS6sNKqY9f8RjeopR624zbaqXUu1Y8JAuLtYJS6geUUs+tyzmUUt+glPqkUqqtlFpqfq5S6g1Kqfcu85gWq4cl7cvD+wC87YrH8JY5xvD1AP7p6oZiYbGW+AEAz63ROX4GQAXAfwR3Ti4TbwDwniUf02LFCF31AB4VaK2/eNVjmAda6z+56jFYWFjgNQA+qLX+V1c9kPOglIprrVtXPY5rD621fVzCA8CHAXzceP42ABrA6wD8HoAGgM8A+Cue/Z4H8GsA3gHgJQAtAL8N4DFjm+dGx3rWb1/j/NrzeO+U8WoA7/IZx98A8CIAB8A/ARCFu2L/09FrzwO44znW+wF8evT+AYD/A8ANzzZRAD8N16o4AfABAH/X/YmObVcA8EEAhwDaAP4tgK+76u/XPq72AdeL9GkAHQB3AfwDACHj/fcCKPrsJ7/z0fzyzpHnjO2+F8BPACiNfqf/O4DIss7h2ec5n+0+bLz/XwH489H1vgzgBzz7fz2A3wLwAO695f8D8F3G+2/zOf7zo/c+DONeNXrtVaNtvs1zXd8L4B8COAbwBeO9NwP4+GiOPgTwIwDC53yHz2O195iXAPwogL8/GpMz2i571b/feR7W0r56/BJcEvoAgL8D4JeVUk9qrQ+Mbb4ewJfBnSAxAP8rgP8LwNfOcZ73AbgDIAfgb49eO5i8uS/+XQDbo3HeAfDjcBcRXwd3UjYA/KPR9XyLsd8ugP8ZwH0AOwC+D8AfKqWe1VoPR9v8CNwbybsBvAB34n6HeXKlVBTA74+u4fsBHAH4WwB+Xyn1aq31wzmvx+IaQCn1JgC/AuAX4f4uvhLu730LwPfMcahvB/Av4RLHz45e+wvj/e8D8CcAvgvAV8BdGLRH51zWOYhPwp33fwzgx0bbHwOAUur74c6nH4FLYK8H8D6lVFNr/ZOj/R8H8G8A/OPRGL8BwM8rpYZa64/AXfj/2Oia6HavzXEdxPcD+CMA/wVG4Val1FsAfASua//dAJ4C8L+M3v/vzjneKu8xAPCdAL4A4G8C2B8d82cB/OdzX/lV4apXDY/KA5Mt7e82XtsC0AfwPcZrzwPowVhZwp2AGsC3jJ4/h3Ms7dHzX8NoNT3DeP0s7QqMVSmAXx1t9x8Yr/3t0WuJCccNAnjM3G903S0A329sp+BaEtp47e0AugBebbwWAvBFAB+46u/YPq7mAZdI/6XntR8AMABwa/T8vTjHCh49L8LHAzXa7jMAAsZr/z2AJoDCMs4x4dq8+2bgWojv8Wz3w3Ctx6DPMdRonvwMgD80Xn+XOb+M18fuVaPXXgV/S/uTPud6GcDPe17/7tEc35pyrSu7x4xefwmulyRlvPZdAIYAXnvVv+NZH1aIdvX4F/xHa30C13q85dnmk1rrV4zt/s1ouzdcyghP8XGtddV4/gW4JPr/eF4DgJt8QSn1rUqpf6uUqsJdlNDCf2b093VwPQi/xX20O6P+b8/5/0MAnwDwolIqpJSip+hfAfhLC1+VxcZCKRUE8DU4K5r8FbiW3TLFW7+px622/xNAHMCzSzzHefh6AEkA/5RzYDQP/hDAHkb3DqVUXin1j5RSL8Nd9PfghtiemXDcRfFRz/Nn4FrIv+ozvhjO/6xWdY8hfk9r7RjPfwPuQmMer+WVwrrHrx4Vz/Mu3B+3iSOf/Y7guncuE35jrXtuZN3R3xgAKKW+Fi4Z/wbcuNMR3BXwn+D0Om+M/h57ju99vg3XfdbzGdtGCf0sloZtAGG4GgcTfF5Y4rm885DPL3Mebo/+/vmE92/DtXQ/DHeuvA+u+70GN5T05iWPx/u5c3xeMjfHNw2ruscQY9+h1rqplHJw+ffShWFJezOwO+G1B6P/26O/Ec82ebiuuKvEt8Ml37dq+s+UetyzDWPRO3DdVzCemyjBFbf8LZ/zdC4+VIsNRBHuIs47R/ZGf/l7asMzP5RS+TnP5T0Hn5vz8KLnOA+8nm/DWcIEgM8qpWKj99+ptf7Hxlhm9ayeuQ649xI/6AnjeweAT/ls/+KMY5gHs9xjiLHvUCmVAJDC6Xe49rCkvRn4GqXUHbrIlVLfAPfH96ej9+kKei1cAQuUUrfhpot83jiOnxW/asQB9DiZRvguzzafhnujeDNcYQiUUgrAf+rZ7g8AvAnAK1prP++DxSMGrfVAKfUJuEKinzbeegvcWOUfj54fAEgrpR7TWt8bvfYmn0NOmyNvVkr9kGH1/RW4cdo/W+I5zsMfj855U2v9234bKKWycEMDHeO1NID/DOMk2x29F9Nat43XDwC8yvO633X44bMA7gF4ldb6QzPuc1HMco8hvlkplTJc5N8O9zO50sJX88CS9mbgGMBvK6Xeg1P1+Ce11r8DAFrrg1G1tfcppZpwJ+y7MW61Aq6Q5s1Kqb8Md2Le11rfX/HYfw/A31VK/UO4Mep/D8BfMzfQWp8opT4E4H9USvVwqh7PYPwm84tw1cDPK6V+FMCX4IrY3gDgodb6x1d8LRbrifcA+F2l1M8D+GW4Gon3AfiQPs3C+B24ZPdzSqkfA/AE/JXlnwHwnyilfgeu4OuzWuv66L003Fjyh+Cqx/8+gJ/SWnOeLeMcU6G1royqmP3EyJr8I7jz/RkAb9Raf7vWuqqU+hiA/0EpVYO7ePlBAFW4c8ocBwD8t0qpPwRQ01p/Fm5myg8D+Fml1IcBfDVcIdks4xsqpb4PwD9RSmUA/HO4i4MnAfxlAH9Va92c5Vhz4Nx7jIEW3HvpB+C6xD8A4De01n4K/vXEVSvhHpUHJqvHU57tXgLwo8bz5+Gqvr8HwCtwf3T/HMBtz35Pj7ZtwF3tvhln1ePbcOM+JSyYp+3Z5r3wqGXho2SHq+S9Oxrb7wN4tc/xY3AtpSqAMty0jvcCqHiOn4WbK3sX7s3gAK4g6Buu+ju2j6t7AHgrXI8NfxNjedqjbb4Vbiy4CeBfw/VMeX+Hr4cbC23AP0/7J0e/zyqAnwIQXdY5JlzX2L7G638NriizNRrP/wvge433n4brmWqM7hs/4J2vcAVYPwI3TWoII7ME7v3pi6Pr+GdwidBPPX5mbMbn8K9H56/BzRP/n7zfiWefVd9jXoKb5vZeuKGFBtzUtNxV/37neajRxVisKZRSz8P90f7Vqx7LZUMp9ftwCzJ841WPxeLRxqju99/Rp3nQFhsGpdRLcBcF5+WKrzWse9xiLaCUeiPcAgqfhKsGfiuAb8ImFT2wsLCwWDEsaVusCxy4Ma8fgusq/zyAt2mtf+1KR2VhYWGxRliZe1wp9S1wY49BAD+rtX7/Sk5kYWGxUti5bGGxPlgJaY+qFH0OwDfDFYV8DMB36k1S6FlYWNi5bGGxZlhVGdM3wO348iWtdRduGsayK/FYWFisHnYuW1isEVYV034MrvyeOIArMhIopd4Bt2oO4KZAWFhYnI+i1tpbKW6VOHcuA+PzORKJvH5n5zKHuDnQWiMQCMDPw6mUwnA4hFtX6Ox+SqmJ+/O9aceYNqZJ2583pnW7lkngMc3z+43Ju+2qce/evbnn85UJ0bTWH4TbXo3pFBYTMO1HO+3HZf4Ip+27yA/0omOaZz9zn0k3gVnOPQ+u4pwz4uXLPNmsMOfzrVu39Dvf+c4rHtF6otfrIZFIoNc7Wz4/HA6j2WwiHA777hcOhyfuz/emHWPamCZtf96Y1u1aJoHHNM/vNybvtqvGu9/97rnn86rc4/cwXhj+1ug1CwuLzYKdyxYWa4RVkfbHALxaKfWEUioC4DtgtF20sLDYGNi5bGGxRliJe1xr3VdKvQvA78JNE/k5rfWkVnIWFhZrCjuXF8dgMMBweNpRclL8dx4EAgEEg8EzrweDwUtz6XrhvU7g9FoHg4HveLnNVVxLIBA44x4fDAYrO9+ysbKYttb6o5jcU9XCwmJDYOfyYkgmk2NkEAwGUal420XPj0aj4UuSk8jxMpBIJMaet9ttpNNpNJvTe4Nc1rXweDyXd1yBwKqczsuHrYi2ZvBTNV4U61Zf/jxF6TRc1rXwPJepJLXYbNDiTKVSGA6H6Ha7Y4Q0GAyQSCSEyNvt9hg5DQaDieQRCATQ6/UwGAzOLAYAl5Ta7fbEY8RisYnWZDAYRKPR8CXKaccDXNHWYDA4Q4KJRALNZhORSAThcBjt9mnnz4teyyLg+QeDAQaDASKRyJn3+V1N8w6sAyxprwGmKaovQuIm8Ux7f94xzTsO83jTUksmEbf5/Dwl97IxKR3GErmFH1KpFHq9HobDoa97HDi1+oLB4MzK6GAwiGAwKK5dL5kFg8GpLt7BYDDV5TzPWMzj8eG1tEnK3W7Xd6wXuZaLgJ+9n3t8nYnahCXtK8YychA3DZYELa4jhsOhPAB/93ij0bjwedbNPT4cDs8sCAaDwVq5x72w7nGLpeFRcI9Pwjq5x6/6nBabDz/rdjgcXoiQTCt9neFdwPjhqq6Flv6mwpK2hYWFhcW1AD0bftY7sP6LnVlgSdvCwsJiAXhj1n6pT48C1iGFip+91hrpdNqXtOv1OobD4ZUr7S8KS9oWjzQWLcdqYeGNWVOZTFGTlxgovgoEAlBKbVRusB9Ifiw3aiIajcr1Xgb4XVDJ7kfasVhMvhdTzb5psKR9yfCShDeOO4korguBrMv1nadmN3FdPnuLi8N0v3pVzoPBANlsFtVqFcBZkRUt0nA4jHq9vlbW3rQFxDTLtNFooNfrSQoY0Ww2pZ73KoqleN3gwWBQUuxMBT8RDoclrc2b7rVpsKR9CbiIQnxdCWPZKWEX3W8WzJtaZ3O0LbwYDofS5MIvD7vf7wuhRyKRMUuT6vHhcHjpxGHmIXsxGAzOkK4JvwYazKNOp9Po9XrodDpj7ycSCUQiEV91+DJgfg8A0O120el0EI1GAbh55OaYSfLBYBCtVguZTGZibvq6w5K2hS/OI6t5FyLrWqxkmsXtzXNfp3FbXD287nHGVAkvKVxlWtGyFdM8VrPZlCIpJmiBXxaSySSCwSCi0SiazeYZ615rjVgsJi5yxsAtaVuci1VZqJuCdSlWMsuiw5K1xTT4pXR5LetltJVcV5iE541pX3ZaFb+LQCCAZDIpVdcIVqgjYW9SXrYXlrQtLh3rRIKzWNoWFhaTsU55z6xENylH/jrAkraFhYXFJYIu9UQiMUYkjUYDSqlrlVO8KCalz02zkE01+ySsskTqZcGS9oZhFRXTLCwsLgdmUxFvahLjrcwpBh5d4vZrJAK4xOw4zpnXSdatVktS7rzK/k6nc67obhNgSXtDMK3RBmDJe5GmKNPe11o/8p+pxeJgPrYf2AHLz9Kmqzkej6Pb7V54HN7a55PGep51uoiobFJ1Mp7TOx5vOh2vn/tzn1QqBcdx5HkgEECj0RBCjsViUkiFYEOTSCSCZrOJdDq9sXFtS9prhEkkoZQ6twjIMlXZV1kDfF7y9ct7n7bvrOPmdpa8LWYFiZrFU7xgnDUYDCKZTJ5Jh4rFYpImBWAs13iWKl5+LmVWCDuvcce09xepIOZtU+rFpOImZpEU8zkwnjIHuHOdC51YLIZutyvH9S6I2u225MgPBoOlNG65KljSvmLMm1o1zT0+r+t83TqMXcV4pi14LFlbzINgMAjHcaTdpJfo6JadVCnMcRxEo1EpSGK2i/TLlfbCm5tMdDqdqelNqxCSMY/am79tvu89J0VkLNbCkqNUezcaDSHeWCwGrbW4xbvdruSN9/v9M6EHft5aayn6sqmwpL0BOM/Sti5yfxX4pPQyvsftp31uj/JnarF8mMTkjduSrGhNbmoe8TLg7dcdCAQkrOCXtmXmjftZ+NQJ8FibDEvaGwabmjQ75nHz28/V4rLhTUuiBfgok/UkzNvq08/SXqfUtIvAkraFBSxBW1hcJ1wXgvaDJW0LCwuLBTCpi9c0UKlNwZRf3JXwpixRvDaJjLyW+zz53n5K76vMF2esmrFnxrcBd6yM+5tjpDW9qTXFZ4Ul7Q3BIgrxdavzfVWw7TctVoFJXbzMAimT9ovFYuIKN2ESEctx8nXGdye5iPP5/Jla5345zZPgjbGTAE319qw4T3F+XoGTRqMhIrZerwetNXK5HAKBAIbDoajwSd7NZhOBQACRSGRMwHfe+ddlkTIPLGlfMhYhCZOw5+1UNQ0XjeOuUxzY77x+47OiPYuLwCQjEghBkgOAXC7nuz/3ndQByyR7M/UJcH+zJpF7QVI3j8UiLlprXzLj8bmvtwEKhVuLENm0Dl9+1+D9bAHI4oZ9sjlmeh5qtRqUUtIoBHAXH5OU9rw+Ks0nfX/rDEval4BZ0rJW1Uhk3jQqc3Ewa26033uLLk4W3WfaeSeRt5mLPQnWSrfwgmTkVSjT8uNrXuIwrehGo3Eml9hrfZrvkUB5HD9405hM8da0XG2SHMmQ5x0MBkgkElBKod1uz0XcpihsHvCzZXocn5Nk2QiEedmsIsfPPRaLSdvQSefn9fp5LTZBBGhJ+5KxCEGYFrZ3/2kW+GW4xzeprOqkz8Na3xazwiQjbw4yXbYkGS+J8rVJBEhSphXpJfRYLCYpT/NiUgoZx9rtdqU/Nc8bDofRbDYvLafZ/Gy9vco5zna7jUgkgk6ng1QqhX6/P3YMs/6432dlNhTZVFjS3hBMssivkmjWqTjLsiqmWVisGkop37SkWCw2sVHGqjFLStVlwk/9HQwGx+avl4BNF/+m52JPgyXtDYOfpW1hYbF58BITq39dJkzLlBa1mdPs13zD4mphSdtibtiSnxYW02HGoInzGnNcFRqNhrj2aaWWy+WxOurrZrl6xXGzpNtdF1jSXiNsmut2ncfmh00br8XmgeItpiABGIsRm7WzvU0zKKCKx+Not9tnXNWDwQDRaFS0Gd73J+1DsZYfzDiymQbFrljrApJ0NBqVz82cz+Z19no9tNttRKPRMTU6O3+tSwhgUVjSXgPMmsY1ycJdVHD2KJDYNOW+F+ukF7DYPJhdvJhqZeY+U4BmWrVeaxEAWq0W4vH4GZc0U5ICgcCZdCXgbAqaWWwkGo2u6rIvDfQIxGIxBAKBM5oA4NSbYdZw5+vxeFy28/N4bIq1fiHSVkq9BKAOYACgr7X+S0qpAoBfAfAqAC8BeIvWunyxYV5feBtdnNfFaxpx+2Eetfqs+eCbglkWNOcp8/1et7DwQzKZRLfbRafTQSAQQKfTGetFTbJm7jMJxY+42SnMBGPPAHwJi2lSwGlhlUKhcC3mMj8XKuBbrdbYwmU4HMpcrtVqyGQyZ1qb8jgUAXqxCYQNLMfSfqPWumg8/0EAf6C1fr9S6gdHz//eEs5zbWES9qJ5wZZYzsd5Ir5H3dK2i/CLgZY2rexEIiFpVIBL2nQ7s7XkpNzpaYVXJuVAm0QfCAQQCoXGiIgFVvz23RTQkuZnx8+IfcOpzPcWojELp6yrtmBWrMI9/mYAz43+/wUAz8OS9lpiFlLadGvzIv3GH1HYRfgFYKZOUYxmkjYfZk1tv9Qm8++s8MbIlVJj+d0ApE81Caxer58hMM4Tk/DniQOzZzU/A3OxwOtKJBLiOZinA1cwGBwrnmKOr9/vS8jBLBBjbrOpixUTFyVtDeBfKKU0gJ/RWn8QwJ7W+sHo/YcA9i54DotLxjTLf5OJzqbLLQS7CN9QmMrvaDQqyvBer4dsNivx4Uqlgnq9jmQyiWKxCK01HMfBYDBAsVhEKpWC4ziSA23mRzN32lt+lXF9kmkqlZJYczablbKkOzs7AIBoNIpIJDJzyVS/hc1VpMxdBS5K2v++1vqeUmoXwO8ppT5jvqm11iNCPwOl1DsAvOOC57dYMkhsppuNf0ninKR8b9Jzk/SnCeq8iwP+z1U6qyNxJc0GAnxYLA0LL8LN+Typ7vajBi8BzUMoXst0UdDNXigUALiWdqfTQafTkXOk02mxfkm46XQa2WwWpVJprPpbKBQa8xQAOFM1jWEAk+QLhQIcx5GQAYViLChD13a32134Wh8VXIi0tdb3Rn+PlFK/AeANAA6VUvta6wdKqX0ARxP2/SCADwLAJGJ/FHERElpm2dLhcCjHIyFTXcluOmY7PHbbYUyJbi+T6IlJpG2K4Oha6/f7ohrtdDpC4IPBwBL28rHwItycz7du3XqkvxjOkXA4PBabjkajMi+8Sm9gvPMUY7RMU+JxF63/fXx8DMB1h0ejUdTrdSFwqtqHwyEikQji8fiYuzoWi0ksvtPpSCctEnyhUEC/30co5NJJu91GLBZDoVBAtVodI27WC+fnQ2GeWRL2oi5sv5j1pijDZ8HCpK2USgIIaK3ro//fBOCHAfwWgL8O4P2jv7+5jIFed0wrUzpNzT1NTHVRUjMt2WAwiFAoJI9EIiH/h8NhxOPxsfeZM8kblFcQYx6b5G6ChN3tdlEul9FsNtFoNNBqtdDpdJaaa2nJ38VFFuEWkDziab9PLkY7nY6QF+ASdqfTQTqdFkJ0HGcsvnyRmCwt23q9LqrrXq+HaDSKRqOBQqGAdrstLmyS+I0bN8ZahZKQ2+22WNz0fFG97jgO0uk0tNaygM9ms+h0OohGo3L9juPI4qDX66HRaCCbzS50fSZMxT4xGAwQiUQQCoXgOM4jK0TbA/Abox9VCMAvaa1/Ryn1MQC/qpR6O4CXAbzl4sO83jBJYxKBzNq9ahUwredIJIJwOIx0Oo1IJIJIJIJoNIpUKiXPI5EIksmkrKS9q1xaEoRJ2nydTQza7TZCoRDq9bpY8vV6/dKK/j8qhG4X4RcHPULMzW6322NpSXzfzJ8mWNCEniSqn4lAIIBqtbpwm0zAdVtHIhGx3kmibK7BXO5YLCaEzraXAES1nUqlxP1tpqDRtZ1Op7G7u4ty2U0ySCQS2Nrakrh5pVKR8/JzAYB+vy9jnAWmZ8Jbs917z2FxletgcS9M2lrrLwH4Kp/XTwB800UG9ShhWp6w37ZerKqlJ0ELm3EoxqR2d3flf07KRCKBeDyORCKBfD4vEzMUCgnhUknqteI5kVjpqd/vo9VqoV6v4/Of/zyOj49xeHiI4+NjOI4D4NQanweL5LM/IrCL8AvCm2NNsRXJxKxM5lfPm929uCA1q6qFw2EkEglxI89L3N1uF8fHx9jZ2ZEFAt3i7LtN65TWNM+XTqfHXNl0++dyOVlUs+c1PQmhUAiFQkGuhV4FtjSl0K3RaCASiaDdbqPb7aJUKolhMAv4eZmGQCAQQKvVGpvr8yjU1x22ItoaYFlpSX7kf5E4NycjrWkScjKZRCaTQSqVksfOzo68l0wmsbW1JQ3sSdwcm3eSkdRNt3mv10Oz2US1WkWlUhH3IcsTdrtdIf+LWtyWsO0ifJlgHJtxadPSNt3dk9pGUsCVSCTGym8CkBKpwOzE3ev18MUvflEU3MBpdTatNVqtlnjSkskk0un0WCnQcDgs6u56vT5maVOZ7jgOAoEA+v0+0uk0CoUCjo+PZW6mUil0u110u12EQiE0m03kcjk0Gg2Uy2UpltLr9XB8fIy9vXG9o9+1mp+XCXoGrwNB+8GS9pphGQTiJe9FiDsajSIUCglh06LmpM5ms0in00ilUkin09ja2kIymUQqlUIymcT29jZisZismk3S7vf7Y6TNVTzfJ2lTXXp4eIh2u41GowHHcRCPxyUPlJb5LNXPLCxWDW8qkimwmub2NQWdJobDobjJmbJFgRoV4MViEe12Wxa17XYb1WoVN2/eRCAQQCwWQyaTQbvdRqlUkrkYCoXGentzTtEtHggEkEgkoJQS4enOzg52d3cl5t3r9VCv1+We0+/3EYvFcOfOHezu7kIphVAohFwuh4ODAzSbTSHzbreLer0u10bPQi6XQ71elxj0vOR7kRDCJsCS9jXGea5g7/tmoYNYLDZG2iRjknQmk0E6nZZHNpsdI+1sNjtG2iYpTyNtot/vIxKJQCmFbDYLx3FQr9clLtbv988ozU0Ct7DYNJBA6VL2EjjFa3yPVdXMWtyVSgWAO39MkZtJ0PyfolHgtMRqJBIRF3in0xGrnnNRa41PfvKTODk5AQARocbjcYmTJxIJfPSjHx2zgJ955hk8++yzODk5Qa/XQ6vVAnAa4mIBGP6NRqNybSwHa+HCkvYGY9Y8ZaZTcdVqxqnNmLOZ6kFLm+5xkjKJeX9/X1zjpmXNuDaFat7JxnGY1rHfhAyFQshmswgGg7hx48ZY/Lter0tBCLrXOp2OuNQfhQILFtcLnAOmEK1arY5tEwgEUCwWRRhGUuRvPhwOY2dnB4PBAKVSCel0WtIktdbY3d2VRbnpuvdmdwAuGVNYFg6HUavVEA6HUa1WcXJyIkp2xtrNSm9+IauXX34ZTz75JJLJJB4+fAjAXaQwfdSbS870M7rp51V7e4Vp3hKwmwxL2o8ATIGGGaPO5XKIxWISr0omk0LgJFxa25lMZiymTeGZ+YhGoxIDm2TlD4dDuYmQtGOx2MTtubhIp9PI5XLodDrY2tqScXFis4mAX4nDWeB3fmuxW1wmvK51P3c6Y7jtdhtKKaRSKcTjcRFkkpwymYyEtVqtlljvw+EQ2WwWqVRKrHEuytvtNrTWCIfDQpT0ZLEtaDabFRc99ymXy2I9t1otxGIx7OzsoFQqycKi0+ng7t274h1gvD4SiYyp6DmXKZSb1lZ0GszuavxMzfNsMixpP0LgyjgWiwnx0nqmdWySthngDONBAAAgAElEQVRn4za0pguFgjzna7TMA4HAmBucNZB50yBpA6fxdrP3rQnG5Ohy7/V62N3dlQVCNBoVpSitBguLqwQrfNFSXqaVx+pkx8fHyGQy6Ha7Mm8jkYhYvXt7e+KV4mKXJKi1FpEZc68Bd5HAucziK1R8M+ZMNXy1WsXBwYFcGxukMEZtal8A19KuVqsScw+FQhKjp/eOrvhkMjlG9o1Gw7e/dyAQkB7iXuW919Jmnjbvad467ZsES9prgssQUZkTO51OY3t7e0xQlsvlxgqkmCrMeDyOaDQqlnkulxPSpOXObZVS6PV6Y4I4KmIpQKGVbRI6lebmzY2WOFNSBoOBkDbj5dVqVRYJ86TA+ZVYNbHpzVIsrgYkERIN22Qug7QHgwGq1aoowZkPzjkaCASws7ODaDQq7moSWzweR6vVQiaTEWIMhUKS0jUYDNBqtcRdzhgz240Cp5Yw3efNZlOes/pZtVpFPB7HyckJ6vU6dnZ2sLe3J8TPe0MkEsHW1hba7bY0EOFndnh4KNc7CV7PBNPs/PK2rXvcYiVYVXMOuqEoICsUCtja2sJrXvMabG9vY2trC7lcDjs7O0LY3sIEtKLNimfcjqtxPjjJWSucNxaSLm8AVKjSxReLxRCPx5HJZBAKhcSFnkqlxE3OxUK9Xke5XEaxWESn00E4HJb80EVg3eMWywSV1QBEhU3rblHi6Ha7ODk5keIk/H0yNp1Op+V8wWAQ2WwWh4eHkk9NxONxVKtV8VxxjlIpTouZsfNOpyOVxLgwp0I9m82KhUzLNxgMolaryaLh+PgYuVwOpVJJFuYUmdHVXygUxDNA8i6Xywu30bTucYsrwTKsb5It86zpiiJ5b29vY3t7G4VCATs7O0LM5rlJroxxcyKZNcXNmuAUhFGcwkIp/N+0tCmKMVfG8XhczgtAJq3p2iI59/t9JJNJUarbzl0W6wYzx5ruY2A28uYit9Fo4HOf+5wIPqkAp8VM8ShJMZ/Po9VqSVEkLqY5V5guls/nJceaVjMAsYTpwmZTELOhB8dAHQnnfyQSETe64zhIpVJotVqyb7fbFQ2MKYalt44VELlQ6HQ6czeh8cvftq05LS4Fi7hozaYenMRM2zJTtrLZLHK5HPL5PPL5PAqFwljjD1OdbnbyMhXfTLMCTicK3WZUrpokTtLmMc1qaCTlfr8/5mo3myxQBANAFgN02U8TwFlYXCWYDhUMBpHL5cRappVKwqOlyZoEtNJbrRZu3bolc4vqahJmOp3G8fExtra2RBQWj8exu7uLBw8eSFiMC/JwOIxWq4VarSY1EJRSKBQK6HQ6iMfjY65vhqOAU2V2IBBANpuVMXc6HZycnEgeNzBeonhrawsnJyciIAVOe4zzuP1+H/1+f8wrYT1e47CkvQHwtq00HyQ9k1T5mpm2xTh0LpdDNpuVWDbj2UzdmtcVxbGRsJnbWa/XpeADbzR0xXlJm9Y7Y+5mOph5XXywhWC73ZZYu1kq1cJiXUG3OUmJBU2YBsmYbiKRQCaTEdc1VdaZTEYsWC6WWV6U+hBWDFRK4dOf/vTMoiuz+xYtcWab1Ot1tFotmZe8p7CkMK+JC/RIJCKWfSqVknsSF9wkfsbVzYUEFez1eh3dbvfauLWXBUvaGwJvrjUnUzqdPpOyRWvZJO1wOIxCoYB0Oi2W9c2bN+V/ilHmBS3nWq2Ger2OWq0mytJGoyFk3W63ZRXN9BRO1GazKW37tNbY2tpCNBodW4wAkHj6YDAQsRurpvFaLWlbrDsYzzU7ZvF1ephY7rPb7Uq5T87PcDgscygYDGJ7e1uKmtTrddGYHB0dzUzY0WgUe3t7Uh6YwjRWPKM6vFwuY3t7G4FAYMyDxuvo9/viEgcgym/G3BkHZxx7OByiUqmgUCig2WxCay0aFVrr7B42D7xu8OvQKISwpL1B8HbbisfjyOVykj+dy+XGyoYyx5HEzQIpmUxGFKTxeNy3CMosIGEz5lav11GtVlEul3F8fCzFT1hzmClgXtJmXCsejyOVSomAxNuHGzgVxZkPWtgXIWzrgrOYB151MnB++UzWyjctbfakJplRs8G5S8Kie5v75XK5sY5hFIExTMQw1SwIBoN4/PHH4TiOKMbpITPBtLFisYjt7W1p80kXN9+7c+cOgNMGKUzrSqfTUtGt0+mgVCrJfYhzPhKJoNPpoF6vS6x+WlMg7me67JVSY33Mze+GCxpvvJvb8PNYZ1jSvkQsolA205JM0mbucj6flzj1zs7OWOlQMw2LqR3cj49FLVTGqrvdLlqtFhzHQa1WQ6VSQalUwtHREarVKhzHke2oLjdJmYUdzJzRaaVIzTQ0b2hglmuYpNCf9N1MWwxYon90MRwOz3imSDJm/jAA6XIFuERWLpelaUa/3xfhJX/TuVxOFOHZbBalUkkIm32vSXxUYDM/G4D0806n09Lxyps6lUgkkM1mJS5+//59AK4I1FxcDIdDtNttqbr24MED7O3toVKpyPuBQADdbhcPHjxAtVqVWulMN8tmsxgMBiJI4/WS7CuViqR8sUoaXfWHh4dT5xlDCiz/Ss8d+5ITwWAQlUpFFhl+XkXua0nbYiZ4f5h+8dxQKCQddlgh7Mknn8TW1hYKhQJu374tRMxCCSQ59sU13eXxeFyeA6f1iv3cSCRcpnFRaNZqtdBsNnH//n0cHx/j6OgIR0dH+Iu/+AuUy2VZMZOIvaVXaRnwJsiewlpr33Forc9Y2dbStlgnePOHAUhs2hR5sWMXyZZx4FQqJbFl02PGzllbW1sol8tIJpPodruIRqNCriTJZrMppUfz+Tzq9brMb87/VquFg4MDIW8Asj/Tr0jWjC+3223J9TbzxLko4QKh2+3CcRzJF2doLh6Pjy2UT05OxrJDKMyjpWwu8ueBn6Vt3eMWK4cpNCO5smIR3U1UfW9vb2Nvbw+pVEpaYpK0zZ68ppuIFrbZWICkytdYaYwiMjaqbzabsgJvNBqo1WoSz6a1XS6XJV/TJEXz/1gsJm45ClnOa/rhFaZdlLAtLFYJWq0kp2q1imAwiGazKfFflg3lNmYRpHg8LhXKzOeMFVOFToLsdDpjFdCGw6GQaqFQQKlUku1NCx2AxJM5BlrtjLG32204jjN2j+G9odFoCKnncjmpbw5gzOtggiI73odo7bJYi9lDfB5wMW/CG87YVFjSXlPwR0yLku5ub6ctM22rUCgIabNKmUlofhapl0xNi5jjIGmTsKnoNNtlkrTNR7VaRa1WmxqTYtpWu90Wl/t57nGvB8JU01tYrBvoRSJRmuIzU5jZaDSkHSWFWKxhQFU5syvooaLgCzh1N5vESD0JrXh6tQizgQ+LsJjjA051JJVKBa+88oqUM/VmrRwdHeHk5ARf9mVfJmr2Xq+HZDIpinATWmvx9rHuAuP4bM1JcZrFKSxprzEo1KKrOxaLSaoW07YoKmP6lllohBWNJpXrNFtaet3W/J/b8IbAVTytY8dxxki7Xq/DcRw0Go2xggqTwLQwLgrYcvO8iWpJ22KTQKuRi1KCsVuCoSEWNqH7eWdnB1/60pfQ6XSkZK9fnHp/fx/9fl9adJL8WSSFaWGAK2Y7PDyUhX6z2ZSFczAYFA8A07Lq9Tq++Zu/Gfv7+77X+JVf+ZX4qZ/6KWm1SdGXeQ+gSA4Yt755fpJ7NBpFo9FANBqduuh/FGFJe03B2K2ZT53NZvHYY48JWWezWdy5c0fStrLZrFQamiV+443zcDLT4mY8iWkcrVZLLOv79+9LiletVsO9e/fELV4qlVCr1WbKrzSteLPsKc8/6TpMIdqsIjQLi1VhWsyUgqqnnnpKrGYuSiORiJCj+XtnTJlCKzbcmAaGmJgpcnh4ONZVi7FmutV7vZ409DAV6/QIkGwZM3ccB1tbWxPPT6In2ZJ8zYUDXfhebG9vo1KpoNFoIBAIyEIeuD5u7WXBkvYVYhZrkmlQdH/fuHHjDJGn02lJm7pIgRG642h9m+Izs2hKrVZDsVhEtVoVFziJmuKVeVbHZtU38zFL1y5v4RkLi6vAtN8eq4V1Oh0RW7HffLfblXgziY0Lb86DRqMxc4ERVl7r9XrY2dmB1lrEZOxZzXF0u11Uq1Wxcuv1uohMH3/88bE2t4FAALdu3fJtF+o9fzqdhlJqLKYeDAZRLpcn7k/hHNO1WJjFLPs6DaarniEFrxDQbFG8yQsBS9prCorBWBkpn89je3sb+/v70g+XaRt0iS/D4qSbmT985mJTJe44DqrVKorFIiqViojNSqWSWOF+KSaznNMk7/PEaN7KcItcN1O6+L/fOcxtLCwIs1EOn09LFbp//z5e/epXj5XwpBCMf5lu1Gq1kEwmJXd7EQyHQzQaDQmZMeWJrzNTg9tysVAsFmXhzdAVt6OobBJYiIXlUln/gUWQWL3QD2aNdBafYbnWaTCFtST4dDotXgpCay2paI7j+H6um1KAxZL2JWIeAqBy3CTtnZ0dsbSZ2pXJZKQiGFeZFyUaErZZS5yuceZiF4tFUYhXq1UpW0r16KziETPX2s/iXjamCdzm3cfi0QXDM7NYbM1mU/Ka2dOZAlJWNGy1WnjppZfk2FRtO44jce29vT3xfgGntRL8Cr0AEPU5u2wxlYxiOKUUqtWq9Mdm564HDx5I+I3K90Kh4OvWNvHw4UM8++yzYuVGo1G0220Ryx0dHQlxekGPHsWu5nVOg9kbOxKJyL5+Fj0V/GZc3fv+uudoA5a01xahUAjxeBzb29u4efMm7ty5g9u3b+PLv/zLkc1mx+oEm6t+s6+0aTXPYpVyX66MScLNZhPHx8c4OTnB8fExDg8P8YUvfEGIu1arjXX5mie3krmoprLddM1bWGw6KpWKEAotXJYgNltPptNpKQUaj8dx9+7dsSIhBwcHUgo0n88jHA5LvwDW9zcri928eRPNZlNqmHNeMnbNNDISN6ugNRoNaK1lLBTBnmcIsJoaC8OYFi1V5Jtgya47LGmvKcxSpWY7zVwuh0wmcyadiyti/j/J9WtataY7nTV/qRJvtVryMFXi9XpdypXywViYea5ZrdNp6m9r4VpsEkwRJb1UvV4Pn/vc58aU2ACk73Qmk5G0p3q9jlwuJyrudrstinOSLeeZ4zhjfajD4TD29vaQTqelAuLh4aFUIAuFQnLueDwui3y6sw8PDxEIBMSLdnh4KAsGVlM8bxHNFppMcWOMHnDvZ41GY2F3v8UpLGlfMUyyMskrGo1KD2wSNx9sVu+FGQc2W2aarmY22zDPx/2YK93tdqWrj1milEIz1hRnQ5BZaxxbWFxXkKhoTdNDRWEV3bWsgLa9vY1GoyGZHx/72MdQKpVknrJ4UqfTQTQaxWAwkEIs4XAYWmtks9mx8wEuOZbLZWxtbUlON3OxufhnzXBaxt1uV5p4dLtd7O3t4caNG2NdA1nnfBpI0KbbnsYE+w5MAnPBbae+82FJ+wrh13aSbSo5wUyiTiQSiMfjE3/UVH+b6mu/v2YONl8fDAaSV83YFq3sRqOBcrmMSqUilrUpVFn2ZzKLuGyVcW8Li3nR7XZFg8IKZHQHM3YdiUSQz+el0tjOzg6efvppfOQjHznTjctxHORyObGuuThm28x4PC5FU0jc3E5rLZY44FrAtPQp1iKBmnUSWN/8ueeeWyi2a87XbrcrbnnWjWBFNT+Y4TTmagcCAfT7/bECMhaWtK8UtKgpjmDOJvMsd3Z2sL+/j729PWxvb0scaxLMlbBZ8N9M22IuNCuRUR3e6/VQq9Ukjs1Sh3wcHR2NucVrtdpM6s7z4BdvNxcx0/bzPviZWlhcBvxqi5O86bIOBoNIJpPI5XKSo8xSxIlEAi+88MJEIqPly5Km8XhcjtvpdKQAC0WoTMusVCrI5XIypmAwKII2NuPgIoA52alUCoeHh9jd3V1YjBUIBFCtVrG1tTXWepSueDZJ8duP9yaGFFhxDYDcsy4yLrMe/KbH1S1pXzK88WWzRClFHKx8RsUn0yHOy5E0YQq6zBsK1eBMz2BrzU6ng0qlIi7xk5OTMdImiTO2TeHZMuEl7mmCOT/C9l77orDWu8UsMJtSML2KtbgJrTW2t7dFzEUiDwQC2Nvbw8c//vGp52BFsVQqhXK5LB3EGG+mQpvpWGbs26xkCED+Mk/cbLbT7/dF1LYozD4GdOsDLukypOcXSjOrtA2HQ0l3MzU6s8Iv9S4QCMBxHPl/02FJ+4rBlTPj1yyUwqpnZllS1uedFSZpmyVI2UygUqmMpWmVSiVxiReLRTSbTXlerValqxct9FWQtlnhbB5Le1FYgrZYFH5NKUg0XIQDbrlQABLmonfJ7Go1DUqdtrBlyla/3x+LobNaGtOWmKfMbRhjZj44gLFYeCqVGiv0chGYCw225ex0Otjb28PBwcGZ7ekGB07vWXT3A6e9sme1tP06fF1kMbJusKR9hWAuNgsReBuCmM0/5u15TZc4LWvmWpskXalUxoj55ORExGUmaVOMxsnf7XalRviyPgevOO68QjGT6qVb97jFVaDT6Yj1S6s6EAggk8kgEomgUqlI33jmd7/44ot48sknUS6XfV3kDJFFo1GUy2VEo1Hpoc2qg2aXL1rW3G4wGKBYLMpin72rgdPuWpwzzMVmFTQv6c1CmKFQSCq38RrpIqeXYVJlt1arBQCS7sbcdABzxbT92qJeN1jSvkJQ1cmGICyGwMIGjG/TfcWUkUmrRrMgSq/Xg+M4YlXTsqa7u9Fo4OHDh9Lco9Fo4Pj4+AxpU6jCRgXAuDKd1+HFrIRubjdPTJuxeq+4zsLiKkBLkETIOcr5ur+/L0puLtZLpRKefvppfOM3fiPu378vrmPGnUOhEPr9PorFIgKBAAaDwRnrmFYzS36axVbM+UMCZZevo6MjaK2Ry+XkfEwvSyaTY8Iws4DJNJgeMjMnnLodM2XNi1arhWAwKJ+BVZFPxrmkrZT6OQDfBuBIa/3s6LUCgF8B8CoALwF4i9a6rNxP+ScA/McAmgDeprX+5GqGfr1BYqRbu9VqIRKJSMs6r6ucKVuTSLvT6Yh4zOyBbZI2U7lYh5hkbxK2xWbDzufVgUIvb9iIrS+HwyGy2ayQEVtPDgYDPPXUU2NWKgC88MILSKfTkgplNtJhhTKSM8VaJGpa4BxLrVZDOp2G4zg4ODjAM888IxY0q5cxrYv3kHlheg55fzLvR5Mqt/GaI5GIFFtaNcyxzOpJWBfM8ul8GMBPAvhF47UfBPAHWuv3K6V+cPT87wH4VgCvHj2+DsBPj/5aTIAZdzbL+JFsHceRHxQVlKzp6z0OXWLMi6RgjMTrdYdXq9WxfGu22zTd6Ksm7E1J3TJX/es+1nPwYdj5vBKwTCnVz6xYRiGUN9+ZbvPBYIBKpSKua9YnZyiKwjHmVfP4AKSmN4mOljjvDyR85nzX63W87nWvkzi7FxfRqTClq9/vy5iZGcNU1qsGLX2m5gH+QrV1JvFzSVtr/UdKqVd5Xn4zgOdG//8CgOfhTvI3A/hF7d7V/kQplVNK7WutHyxrwNcNJmGzEplZ0rNUKqHX66Hdbkv3GypQTWit0e12x8ifaR10b9dqNSFkpoaYcWv2wGY5w16vt3KCMgl7WeKyZcLPRbfJBG7n88VgqpxTqZTc+Kk/6Xa7Y+7hSqUiRMreALRiWWWMVl+tVhOSZ5cslhSuVCqIxWI4Pj5GLBYTzxtj2TwGVey0bkn+rDP+2te+9ty0US4OvGGw8xCPx2Ux4VWuc8Eyqe43609EIhFxxTO7houZi2AwGCCZTMp9zQtzIeX1eKwbFvVD7BkT9yGAvdH/jwG4a2x3MHrtkZ3kwOTcYRItcPojZ4cbqk+LxaJ0x2HBBjPX0CxDSmEKY9tm7qNJ4nzUajUh6G63K5Y5V/iz1g+/SI60GZv2xqmnteb0K8KybLKfpcDLtC5hGwQ7n+cAY750TwOn85duZqZYUgXdbrcRiURQKpWQzWZRrVbFc0ZRmJnp4fWcBQIBIV/WDCe5MVbN99l6k3FsZqicV+cBOCWqVqsliwoS7nkIhUKSCuotV8rPwO9Yw+FQhGgUrtXrdTiOI672i1jppoueMXtTYc5iVqy5TnJfV1w4eKC11kqpue9YSql3AHjHRc+/6eDkBiAkxQnOFbpZgIV1g021tXksk/hI1rS+SeIkcpZU5HZm8ZVFSGhR4vLLu552rPOqpl2UQK+bdT0PljGfJ7lerwNMYjXjonrUGS8SiaBcLgtpkWSpR+E21JyYSm8uqofDIRKJxJiCmsTLkqjFYlGKqlAoRmGrmbrFjJR6vS4LhPMsSL5PK36emG8oFJIKa1w4MJOF4jovtNbS/pOeQR6r2WwuJcadSCTEIPGK7biQMBcT86SYXTYW/TQO6SZTSu0DOBq9fg/AbWO7W6PXzkBr/UEAHwSARW4SmwrvzZ/WMSe91npMzMEfLV9jeb9ptce9RVVMEjcbGvBmYRZfMQn7okVK5oHpIuffqybH8zwHl/n5rBhLnc+3bt26Fh/KeTBv/IFAYMy1aoq52GaTsV1qSmKxmPS25m+tWq2KV4wEGw6Hxbp2HEcqo5Fo0um0pG0xG4X3AvbmZmcvk6imYdHUKc4HxrRpkDDEN+m+1W635S/vU/xcLyoU472PFjTvc/x+6A1h97VJaWnrgkVJ+7cA/HUA7x/9/U3j9XcppX4ZrmCl+ijHv86DGd/qdDpnrGc/i9JrXU6zAL1Wqzen2S/PmS5f/p1myS6DsMz0Lq+bexFre9kk6ufRWNW5rhB2Pi8Ar8qavwfW8C6VSuLardVq0skLgFQHI3nT+8WFANM8uXgPhUJyv2AqlVJK3OTM32ZlM+ZFk/C4aADGS3kGg0Gx3huNxkLkSEOA8XumpgGnLTk59knueY6d79ODEQqFUKvV5h6TF6ahQqueoBez2WzO3CP9KjFLytdH4IpUtpVSBwDeA3dy/6pS6u0AXgbwltHmH4WbHvIFuCkif2MFY75WmFfscZlYldrTJEDeQLzEbbEa2Pm8GvBmH4vFcP/+fVGOk9CSyaQ0+qC72GwwQvGpWR6YDYAYvmJ/Ai4ESMiMYQOQXGimfwLAyckJ+v0+9vf3xbvGc7COOcc/K2nzGPQWsrBKIpGQ8qWM79PFP63KoVlm2bS0Lc5iFvX4d05465t8ttUA3nnRQT1KWGdr7SICMy+83gFOYLr+TfKe5XxeS9uS/Wyw83k1YNOLYDCIXC63dmUzzTRSM92JoCVKTHJJcxsuDnq9HuLxOOr1OrLZLEqlkliuXGiYIttJ4OKBud6xWAzlcnll90fz2sywwSbAVkS7QqxD3HYavK5yL2Z1S3tdzLQqGK/zEvcsJOxnna+SuNf5e7K4Gpg3fjNNax3ykb2gwJXudFZuI6ia5tiZuuVH3BS2mUp5HouvMb7OZiGRSGRqvwLGv6PRqNSPiEajUkN9mZ8DQwKmepw55bwHrav3E7CkfenYtJv/JOKeN8ZrkjGJOhwOI5FIIJlMIh6Pi2J+FuJmnG9eC92L8/aZdk2b9l1aXBymBertdDctZnvV4KKYlnaz2ZTYrp+iPBwOj5UUJQaDgYjnGM83RXYkQNZDJxG2221kMpmJqWO7u7uSBmfGmGmxzwrmY/uVYaVY0M+qpqeBrVCvdcqXxfkwb+7X2Y07SU1turB50zObpJjlC2e1ts06x6t0kfuFCLwWisWjBVppZkes82rlrwtMD5VJxl4ypYDMW3N8OBwik8lIzJzHcRwHkUgE8Xhc8tfNdC/WN580X/L5vNRCLxaL0o+BKvpZYabkea+b3c+UUmOFcQKBgCjGlVKyKFlXWNK+QmzKDd+0tidhlvd5szB7iJO0TeK+yM1vmcRtydnCCzMVil4hAFL8ZNUwK7LxvOa4ZgHzw0lUrDduHpd54l5xmpk6SguWgrlEIiHkmEgkRMHOYinTao8z5ZX/E9lsdmmuaubLB4NByUHn9XMBxu91XXO0AUval46ruPl7iWzRwine48yTajXJyja7ANHSnsVqnlSQZdmf73mWtsWjC29cdFU3epIZY8XelCXTDc2xAP5EbhZOMdXtwHje+SSCZRVHM4ecArxgMCjqeOZd02JNp9PnurmZX05hGPsfsFf4RWAuthqNxthiwK/2+DrDkvY1gR+hThJo+RUwWSURmaIzpsIkEgnpG57JZJBKpWR17teWzzs+s6kJ+3uvUjxiidrCCxLVsuEXl6UV22q1kEqlxgqDsK0ma4+fVyCEY6angAsOisi4DYsumRgOh4hEIlL7nLFfxqJjsZj0v+Zx2eSE1eB4HLNc88HBARKJhDRWCYfDSKfTM7cFnRVmmpyJdXaHe2FJ+5rDj2wWrTrmdRfPan1yW/PmwMnKiUs3Od3jdLczj91b6c0kbbNcqyVXi02HNy5ruqkbjYa4s0m+TOcySxPPAnoK9KjOOa1jACLI8lqeZhcx7seCKlprmY90jZPUo9EoHMfBYDAYay9cr9cRj8eRSqWkTGur1RI3PLuGWZzCkvY1xaQqaN7XlnGO88AbAWsjU2QSj8eRSCQQj8cRi8XG1OPmOcySrFR4sic4u5LZggwW1xGcC2YqGcWajuNIVy42DZk1HustU+rdhx4vL8LhMIrFIra3t9Fut5FKpcbc47lcDsFgULwBwWBQ3OlbW1tjLnw2CWE3slqtNnZOpnzRcrdwYUn7GsIUjq1LLjjHREs7Ho8LaXutbdPSBiBNTsxGJ47jwHEcNBqNMRf5OlyrhQUwOS48q3iMnqlms4lcLjdWLtTcP5FIXNhNP2utcfbmppCLNc8pXKPAtF6vI5PJSJ10pRRarZboVmiNc77WarWxWLnpXZhnMT5NWzCta+AmwZL2muAi+cJ+768beTE/Ox6PI5fLIZ/PI5fLIZvNYmtrC/l8Hul0GslkUqxtWtl03bXbbbRaLZRKJdTrdZTLZRwfH8tzU9NA96MAACAASURBVBE6K9btc7LYfJi9m/1+j2adaz+wEAoAITc+94rQrgKJREIqnwUCgbG0TcAldqaLZbNZ1Ot16fzGeLVZqrVarUoxl06nI8K6UqmEaDQ6dU77qekn1VBncxW/RQDTzdgNbZ1hSdviUmCmeiUSibEH3eNUkXM1zDQQ9vdutVpoNBqoVquo1+uoVquo1WriIp/F0p5XST9tMWUJ38IPDOcsKoxcd9Lo9XpoNBpSGY0CNLbVpGiO5MySpwAkB5pufVrdnU5Hwl4sgMIe1+e15vS22ZwmwvNWfiPM72zdP39L2haXAjY7MGPZyWRSqqGZpE2iNFuKdjqdM6Rdq9VQr9fH4tqWSC0sVotKpYJQKITBYCDxZnoEGAunm5wiNSq2WaecxGiK7uhqZ7/xbrfrW+DFhF8xlWnEa+pjvNgU17klbYtLASd0PB4XsjZJ28zVZjzbLC1I13ij0UCtVpOH4zhotVo2pm3xSIIExAwKc94Ap6I1LoYv2k2v0WjgU5/6FL7qq75KrGJa2ixSwtrdJGmzvCu9bd1uV0qUJhIJscD5Ho/b7XY3hkwvC5a0NwCLEpG3xCefUxgyaSJPio8vWoc7Go3iscceQy6XQ6FQwM2bN5HP55HJZJBOp5HNZiVPm40MmHfd7XZRq9VQqVRQLpdRKpVwcHAgVnatVkO1WoXjOGMpKxYW6wQKpEzV9zxgD22maNH7xO5axWJR8pxJ0oztOo6DWCwmIjD28GZJT5PUmXLZbDbHxsjxv/jiiyJE01qj2Wwim80iHA5L1TNayul0GlpradVp5mnH43EArsiUsX+z7zfTyljKdNJn4m1t7O1WZo7/usCS9hrjolajmRvNYvmsSBYKhWQiczIzdjQtt9t8Po3MzdcikQh2d3eRzWZFgEbCTqVSku7FGwYnYb/fF2FKo9EYI2la2STrRS3tSRaHtdgtzoNf3JoCSu9rJEu/ut+zgi7jSCQi+dEsqpJOp9HpdCTfudfrYXd3V/Kkzd95NptFr9dDPp8Xi5YVyGKxGBzHQaFQGCO6druNUCgkBU+i0SiUUsjn85KbzcJJ7ABGMK7NSmn87Jgn3mw2EY1Gx+Y+ld5+XdPMXt7xeHxM8DcYDHzV9DzXdVjUW9K+ZMzaHWsZpEGrmi6peDwusaZIJIJMJjNW9zsSifh28vKSs1noxHw+afzhcBi7u7tiVRcKBVGKmzna9ABwclHY0mg04DiOkHalUkGtVkOz2ZQc1WmkPa072aTP+To3drFYDrxVywDXHc2ULIJNKGhlLgrGbUly1HHEYjEpzam1FgsVgBQxopKbJLm9vS3u7W63i1QqJa5q4DQubHrgIpEIstksAMhigW5vk5R5f6Fru9vtIpFIjKV5mR4/AKhWq2c+H+Z++xWL4fkZ9/YK0bzkTMv+OizGLWlfAs4jgFXVtSZBM5acz+elfGgqlcKNGzeQSqWQTCal/KDXnW6Oycz7Nle7XgI3twXciZTNZiU3e2trS/KyuZgwO3x1u130ej20Wi3U63UcHR3h+PgYh4eHODo6wssvvyx5nZ1OB47jyD7eyTrPZ29i3VPoLK4GZolRP3JgXJeWK/OML6pIpiVruq3peo5GoygUClLUJB6Po9FooFAooFQqAXCJfmdnR+ZVOp0emy+DwUAW9YVCQWqTs2gLCfLGjRsYDAYoFAoSk87lcmg2m2NGgZnSBZxazqyG1m630Wg00Ov14DgOKpUKdnZ2JN2Liw9TVGam0tE9z2s1U778CsyYaWBmoxPvd7sJbnRL2peMeftQXwTe6mOZTAbJZBKZTAbZbBb7+/vipk4mk0in02daXppj9CNtk7i9LnP+HwgEkEwmpUkIG84z9cPsoQ1gLM2Llna9Xh9zj9frdRHfMN1rFZa2JWsLE6ZaeZKlzZQjqqiXgWq1inQ6Lcft9XrIZDKoVCqiuCZZBQIB7OzsIBQKSVEWs/4BFd9mvjRrirfbbSilsLW1JTFjFjyipcxr73a74gZnuI0xdZYeJemysArPWSwWZbtQKCRhsEQiIQsFLnbMMfKzZzEXWvEmaTNWb4LevFV8N5cNS9rXGCZhk5TT6bQUN9nb25MCJ6lUCtlsVqxs03XlR8rTLG3vtkopRKNRmbRMC/Hrm001LKufMdWLVdBI3hS9cBKb45kHNqZtsSj8+jYD01OOFkGj0cDnPvc5vO51rxO3N/OiueiNRqPiFmfcOZVKSalQxq9ZH5zpVb1eT0qR9no9caG3223k83mZg6FQSK6V8XQA0mQkmUyi3+8jlUpJ689kMolKpSIud6rBm80mUqkUqtWqkGir1ZIYttmnGzg7F832oHS7m6Qdj8fPWMy0rDchD/s8WNK+pvCWCiVpZ7NZEYRtb2+jUCggm80KmZvucS9p838AY1Y2n5vbeuPcpvubaV0Ej8WyiOaKmkVVGL+mKM1sdkCLwBKtxXVCr9dDpVLBCy+8IHW7Sbi0WJPJJDqdDtLpNGKxGHZ2dsQ6zmazQridTkfEXlSKs6c2red6vY7hcCgka5YUZoyac5fes1arhUwmg1qtJmTOGDoFaIA7/0nIw+EQ1WpV3mcPATPXm+IyFkOZtDjyistMd7932+sCS9prgkluXdMi5erTj1j5NxQKSRx7e3tbYtapVAo3b95ENptFLpfD1tYWHnvsMeTzeSFtU/FpjosP7zmXcc2MldG69rO0TSu7Wq2iWq0utV2fhcUyYBYV8cZMvSRSKpWEhGkpOo6DSCQiArF79+4J+Xa7XRwcHCCXy6HdbqNSqQhBDgYDRKNRIcRwOIxbt27hE5/4hKRLpdNp5PN5BINBqThGa7VcLou7mu5lMxZvZpsopcSFzVxrqs6pWtdaI5VKSboZw2HxeByO4yCbzaLVaiEYDKJUKiEWi2F/f39M9W2K0mbtWvaowJL2FWOahai1FpcyC4+weAHJ3EvqjGNTnGJWHqNqmyTNEqIUhJkwu2pxxctJzUl8UfjFznnd5irabBbCOLeFxTqBAkrg1BXr54al5ZvL5dBoNJDL5VCtVhGNRpFKpaQ5CF3fnGu9Xg/xeBxaa+TzeZmv/X5frO7hcCjkDbhx8HA4LPX9B4MBOp2O/KWwLZ1OIxQKoVqtLkyQFOCRvKvVKpLJJABI8SPG1QGIq5rCups3b87ttp4nde46dQC0pH2FmMWlyxUqJwRXrZzMTJXiD5ivcYXOmDZV2xSi0Z1m5kcTVMYy5kVRiXl8rqSXAT/r3YyV0/KeJjabFdaFbrEsmL9/Fk3h6965YVrVJNZkMgmtteRNZ7NZiUtz3oVCIWQyGbGmScyMSWutRaBGQqYrOJFI4NatW4hEImLZM6bMWDTd2Ds7O8hms7h3795CXizGmQOBADqdDrLZLPr9vtyXGKemG7zdbovmplgs4ubNmzOfyxSv+in4/cCY/yaow8+DJe1LxrykEY1GJU0rmUxie3t7YroUyZx/U6mUEHg0GpW0K7rLKQ5j2VCllJQMLZfL6HQ6IlABTl3vFHpwVb2Mz2Ta50IrguS9yPGJWV37ltwtzoM31YjEMAnVahWFQgEAxE1N8VU+nxeBFt3X9LDFYjHUajWxXEn8JF7OSbqz6ZF6/etfj6OjIxweHqJSqUis2Uwb48K9WCwim83iqaeewosvvjixCtk08Jqq1aqIxMy0TwpQ2WSExsHDhw/xFV/xFROPy4WIuSBpNpsYDAZIp9NjLTfpFfQSebPZvDZudkval4BJpDELMdC1tb29ja2tLTz++ONjudUskGIqSE3Bl+k6N0mdZA5AVvXtdluqjB0cHMBxHDSbTXFHx2IxEbTdvn1bUsji8fhChSPMieinOPd+hrOKzVbRmesyU/Us1hck2eFwiFQqNZaPPM1C7Xa7IrACIAtnirEoBGW9bjbkYL7y7u4uAHfhzJxjHotxbIoyw+EwyuUyer0eXn75ZWlzCbgEyDmdz+fHUq1Iavv7+3AcB9VqVQRhZvc9ak9MMO/7+Ph4TIBGkRzj/Iylm721y+Xy1M+c5yfRM7XNjMubudzcx0QikRjracAwowmm6627utyS9iVjnht9KBRCPB6XCb21tYUbN25I+c9EIoFsNjtG2rScJy0OzLi01nrM5WzmQh8fH0upUCpWORa6v8yULVMkx3PSegfOkp4ZszZJ24RZF90MAVzkM5/X0raV0SxMmAU6zP7t5/02T05OZOGczWaRSCRQKBTQ7/cl3Yk1DPhbHwwG2N7eRrVaBYAzxVpoZZN0WWCI85MWL8uDdrtdWZQTzNnmgmIwGMBxHHQ6HVGim3XO0+k02u22LDg4T7iQYY10xs1Z1tVxHPEiBAIBJBIJVCoVBAIB5PP5ub4DpZQUV6GlTUzy/NErQm/FpE5fm5ASZkl7jcFcR4rH8vk8dnZ2JCadSCTGLG2zFChw6lYyBV0mzAII/X5fOmdVKhUUi0UpYsKbSiKRkLjZ7u6uWO6MdZuiuEmKc/M9Pwub25CsTTEOV9sXgbWOLZaFSWIzL9rtNv7sz/4Mzz77rMxJKsYZaspkMtK+kjoUKqxZP4FExXxrxrkBiAVKFzQFb/yf+5K4WUiFcXOSmFIKg8EA9XodJycnACDpXsyr5iKARU68nwnj7RS6mdobXrcp2qMYjyVSzwMXP/QazBLT5ri4GNpkWNJeU7D4fjqdlgpmLIqSyWTE0maDAJKaWeKQrqN+vw+llHTOInHxOd1MrDbGjlqVSmWMtDudjrjrGo3GmPI8Go3Keya8JVE5tknubi9hM05mForhDc3CYp1g1hRotVool8vodru4e/cujo6O8JrXvEYsT1rWXFCztCi9a/F4HLlcTuLQZq/4crksrm12zspkMmOuYlq0oVBI8qsZpyZ59vt9ZLNZITHG0gHIPYUiNi4KAPe+wbGbbmmG0czKaSRXLhhYh53Kdi7ef/3Xfx2vfe1rZZ4z8yUej0teOsHxm16PWXBdcrUtaV8xSHq0lhnP4qR96qmnsL+/jxs3bmBvbw83b96UeDZd4RSe0HImGfI1PhhT4irdzI/udrsol8tiab/88stjLS/NCkv9fh97e3tjCwQK4bhwMFe2kwq2mGROMQmLPpjCs52dHblB8D3HceR9Kt2ZDmataYurQKvVQiqVQigUkm52tHgfe+wxABDiYs1tursB4Pj4GLFYDOVyWcqPMlYcDodl7g0GAxwdHcl8zuVyqNVqAE4tSgrB6G7nuZiFwth5OBzG3t6e9B4YDoeiYme3Lrq7vSGARCIhDUCGw6EUcGHddeZzs1VnIpGQcqj02imlkMvlcPv2bbz2ta8V7wVj4EopKWtKXBdB2aKwpH3FIEEnEgkkEgkpN0jX997enlQuy+VyY4QNnOZAdrvdMTUr3VckbE4k02VukjarL3k7aTUaDWnLB5ymoLEqGRcdZq9fs2oS4J+PzddI3hwTJ7XpAcjlcmJp0OJ3HEdSR05OTsT9Z3O4La4KdAUzbzqVSiGdTqPVaolFbP7m4/G46EloOZKQWEioWCxie3tb4r9MmUqlUmKBM+ealjcAcU2zyhnJlL2yAYwJUVkHnGRbKpVkPpq51TQu6AnjXI1Go7hz545UO9NaizHS7Xbl2gKBAHK5HF5++WWk02kxLHh/46KG9wN2TGNam8UMpK2U+jkA3wbgSGv97Oi19wL4mwCOR5u9W2v90dF7PwTg7QAGAP4brfXvrmDc1wbsicvyoqwMRNf3/v4+tra2pNxoIpEQAqVLmwRaKpXOxKnNSmOdTmcsxm26zxnTZhvMcrk8ph5n6cNIJCKkzRrDvAaC5Mr4NTBZ0KWUElEcbxKM09FjUCgUxEXGHFXHcdBqtdBoNOSaF0kHs1g+SEJeLCoknIZJamGeD5jsPp22L/efZ7x3797FM888I/HbbDaLZrOJmzdvilgMOFVas0YC21keHR3JuCjeAlxvHOcaAFkENBoNcT9zUcsGOlRLcxHLxTo9UaxBzjHu7OxIKhYrnDHGzW3pqgfcVpdmPLnX60mHLlrgTB9l1zBa2NVqFdvb2wAgaaNUc7OzGNPSjo6O5haqrRvMVLVlYBZL+8MAfhLAL3pe/3Gt9Y+aLyilvhzAdwD4CgA3Afy+UuoZrfW5kf/LbtxwkbSgZaYURSIRJJNJ5HI5bG9v486dO9J1K5FIYH9/XwidxVIAiDvYjD/fv39fBB6cwLRYObHNPEa+T1JnDeBms4lyuYxWqyXlCnkjYt54o9GQyRkOh0WFStDamPWmZ5Zs5c2Cj62tLakGR09DvV5Ho9FApVIRV/l1sbJXkbJmHHvli3CKFr1YVelZv25bHIdZ8GSefYH5x2vmAvN3zOqDWmuUy2URcUUiEVmkkkwBd94w3HN4eChqbJYXZalQWstcDHDBynri/J/zvlqtimuc85LHNOPhtMRpxZsNSUjcXJR5iYjX0el0pAIa3fxm/2sei94C3gP39vZQrVbHLHiWa910+M2HRXEuaWut/0gp9aoZj/dmAL+ste4AeFEp9QUAbwDwxwuP8Jojn8/jxo0buH37Nu7cuYOv/uqvRqFQEIEZcxxjsZgUVGC7ynK5jM9//vO4d+8eHjx4gM985jMi0DKFXiZR86bvVW/T8ubDrIjGGwdj15FIBOVyWeJnzHsk+fOmZLoCZ4VSShYCDBOwKQIfrEVeLpdxdHQkrsfhcIgHDx6s5Hu6RvgwVrwIn2RpryKdxgyj+CGRSKDZbPqe97x95x0v5yldzLu7u1LQKBKJ4IUXXhA3MYuhUD0+HA5x9+5dAMCDBw/QbDbx4osvAsCZ+t7sSc392KaSJErXOK/dbDDCMBLHCEDCa81mU4wCHo8EygJNAFAoFFCv16d+Fpyr/A7MlpusxsgCTYPBAE888YSkZB0eHkre+3A4xKc+9Sk888wzM38P6wbeW5eFi8S036WU+i8BfBzA92mtywAeA/AnxjYHo9csJoBikXw+L3nYbPRBgmTKE4ulcMVKhWqxWMTDhw/xyiuvjFUyMgmaf01Lzfu6aZUz3m1WIzMfJsGb5U699cpnLYjihWl508o2a5A7jiM3mFwuh0qlgmQyKZ+PhT/sInw1IImZxMSYMb1mJFHGpm/cuIFUKoVyuSyWrdk85P79+wAgos9arSZtNAl6xhgyM3tG0yo3LXpauZxHnGf0VrFwCWPsXDAw/QyApKURwWBQmn2YWSJm4Rk2FxkOh8jlcuIBYdiN+daM19+/f1+EcwcHByv5zjYVi5L2TwN4HwA9+vtjAL57ngMopd4B4B0Lnv9agIpNpnSxXebOzo4UAfCqr0lcJO1arYZyuYxSqYSjoyNf0p4VJsGaVjmAsTGY23qLpHj347bD4VD2nVRMhefxVkajKt1cBNBC6Xa7yOVykgbHeLvF3LCL8HNgLlC73S7q9br8/9nPfhZPPfWUkLbZLvOJJ55ANptFsViUedtut1Gv1yVbJJFISO40leAPHz7EcDgUERaPTRGY2TrTW4CIpUmHwyEKhcKYEM0UsrLCG93g1K/Qw2aqxummZjhhd3dXFOEkem5Db0sikcDJyclY/jbDbUwZLRQK2NnZwcOHD+VzMb0C6XQan/jEJ0RZznLKXGAwbGe+75dDf55GYVNqky9E2lrrQ/6vlPoQgH82enoPwG1j01uj1/yO8UEAHxwd49rm6JjKaU42kh/Jmo9MJiOFU/x+XGZ1Iq5+KUJjpaNlirFMoiZxmh3H+CP3XpeX4AGIStaMrXstYjOnm/9Tjet1sVPMw1Z/fORyOfmsgMl9vv3e98Mjkj621EU4v4NFYIrDzhOSLRvmPDXHQrAftdYazWYT/X5fhJp3797F/v6+WL3dbhfFYhF7e3vIZrN4/vnncXBwMKa2fuKJJyTliwKubreLQqEg8fQ3vvGNuHHjhnwOs4aazKYaPIca9RbgGFqt1lhtBy5I6DansIwESEs4nU5LX/uTkxPkcjkpg8rFAC1piuYAiA6GCvNisYhgMIhXXnkFb33rW3H37l0cHh7i4OBAslc6nY4UX6HanN8F5y4br/DY0+bstAYjm0DYwIKkrZTa11ozePjtAP5s9P9vAfglpdT/BjcG9moAf3rhUfqPYWq8lO8t6pr1O96k85wH1udm7jVXidlsFs8++ywee+wx3L59G7dv38bu7q5v3W3GiCqVCiqVijQCODk5QblcRrVaXSphM4bF2HIulxNvQKFQwO7urqRpcPFhXictB95wqfZuNpsoFotjfYRNS4E3Ly4OotEotra2zjQnCQQCokB9+umnEYvFpHIUyzfypmpaSLzZ8sE0mcsk58sWXZ6HZS/Cb926NdeFMB/ZDLsQ/B2wGMc8BE6y8DbyYHEgE0wlNK0x77kePHiAfD6PXq8nRY7M4+3s7EjPauY7v+lNb8KHPvQhFItFAO5i886dO7h37x6Ojo5EwEnxFmO/APA1X/M1+Nqv/dqZr3cS2u22qLhZipQFVzjHvdUG2+029vb2xPLmdxQMBscalwBuaODhw4fyWWxvb6Ner4+p2GlwHB4eSuw6EonglVdeQa/Xwy/8wi/g7W9/O46OjrC1tSX7cl4//fTTslgC3HsqFwjMDzfLqHr7mZvf5yoyGMxwgXm+VXj9Zkn5+giA5wBsK6UOALwHwHNKqX8H7sr8JQD/NQBorf9cKfWrAP4CQB/AO2dRji8Kb94v8f+3960xkqVnec/XVdW3quq69r1nZmdnMLtee8HGOJawIHKkGPzHsbgY/yCQIDk/jBQkIsWBP/xBIlECIlKC5AgkiEgIEkT4BwjDysaJ5cVZG6/Xy3h2Z3anZ/pW165736rr5Ef38/Z7Tp+6V3dXz3yP1Jqe6u5T3zl1zvfenvd5+yU/DYpeNlnHcSQNHo1Gsby8LAzoWCyGlZUVLC4uIpVKidHxgim1vb09USyjJ0pPf9SCA1QfI+OULWeJREKEI3R2gKlp/v7U1JTU9MjsphLS9va29H/rupceKUrSHY8J+Ke3IpEIUqmU1Oc46IAlhEqlIulIPsgUYWHUcd1lDYfFVTnhvO401NQR0JObmILWwzb62XQjkQjK5bIrumLEqCMr7Ti2w/r6Oqanp6V+y/uSIzDprPL5TiaT+PrXvy7DOqampvDcc8+hVCpJmxeNdLlcdrVQNZvNcx0Zg+D4+BiZTEYEivhMMrXOSWGUP+X11wacAi53796VvulisehislMeNZlMikjT3NycRNwzMzPyOZNkxlbWTCaDarWKV155BS+88ILIp3Ii2tzcnJDgSE7VzzizbjobcNHPtCb+6YElGpqANkoGfC/s8c/4vPx7HX7/NwD8xjCL6oZ2xlq/pmuio4hgdC2337+j0U4kElhZWZGolEY7nU4jkUj4CghwoABT4MViUVq8aJBIGhsV2LITDoeFREPNcxptHWVHIhFEIhEXw11P/WK/KBnfW1tbomPMtJ0e+0nmK8eI6vo+vyemp6eRSqUAnBFvqtWq9Ijm83kpIdTrdWHOMhVKUs5Foxcn8jIczctwwllO8cLvHg2Hw7LJhsNhYTHTaHN0ZL1el7SsBg0toywNzbPQm6rjOAiHwy5ylVYmawe2PwaDQRH9YZRFp7bVasl9eOvWLbzyyiuyuc/Pz2NyclJq1jxPOhCzs7OoVqvyPIwiImQaem1tTbTKGQ2zDSwcDkukzfXwM2T2a3FxEffv30cgEMBLL72E3d1d4apQl5ydHGtra5IVZOqaJb39/X1JUWcyGbzvfe9DtVpFo9HAo0eP8OEPfxihUAgLCwsol8uIx+OYmJjA0tKSPNsTExOuzAwzE9oJu2jVNN0q6HfvAWf3Gb/XpZdhMDaKaIMYw3b/15F2p7R2J8GPTuvrZ62Tk5NIpVJYWVnBysoKfuiHfgjz8/Ni8BKJhMtAelEoFCSyLpVKePz4sfRm82dM+Y4CwWAQ0WgUi4uLEk3HYjGsrq5KzTgWi0k/OTXQ5+bmXAx3jWaziWKxiM3NTTx58gSvvfYacrkcSqWSrJsedDQaxcrKijgD3AiTySSSyaRslhpra2tIpVLY29vDrVu3RPu5Xq/jyZMnruzEzMwMarUaqtUqarUaDg8P4TjOpWqZ95Iev6hU+WU44SQReeG9L7jZMYImUYkbItOLfJ26Al5jxvZEvxSl34aq10ED0ouQCo3O5OSkZM5otFdWVqTey3aqjY0NSYsDwIc+9CF8/etfl2vDXm463WxNo4jRsI744eEhHj58iLW1NQAn0S6JXezFjkajKBaLrglcrVZLsh7T09NIp9N48uQJDg4OkEwmZVhRvV6X0Z0UYqKjTIEUfgY6NX90dDKdjFMF19bW8NZbb6FareJv//ZvcffuXdy7dw8ApCOEwdfCwgIODg6Qy+Vkz6Dgkm53rVarQ3ErukG3CrYTT9GcDN7Lo3DExsZoDwpugH514HHA1NSUGJ9kMonFxUUsLi5KTYz1W2/dDYAQrbS0aLFYFCNUqVRcwwS6tTv1knVgWpyTxTikJJVKYW5uTpwNPWmMEXa7G5J932S6ZzIZZDIZmfnL6xQOh1Gr1RAKhSSF3Ww2RVyF0YD3YeR7M8JnKYG66XqueKVScTHU2SeqjYfFcGgXabcDe40Z7dKYaZ4D66pe8J5rd9/71RvZx8x2p1420nq9jng8LoRK3vO684M16mq1imQyia997WtC7Lpz5w6+/OUvy/EmJiZQKpXOtU+R6MZsnFZS0+dE54Fzuv3O4Stf+Qp+4Ad+QAwy18ZhHtFoVFrTgsEgYrGY9GzTGGazWTx+/Fiu4w//8A/LZ0tSbLVaRS6Xw6NHj+S9d3d3MTMzI88UzyGXy8l5hkIhPHjwQAKCcrmMBw8eYGdnBzdu3JDOEF2vLpfLLvlVrSdBYmy5XMbe3h7efPNN3Llzx9eZGwbedDizK17wPtOs/lGkya+90R5nsGbFtq54PI5kMila4uzFJvtag1riNNjlctllrPWsa61m1IkZyZ+3+x2KMwgEAAAAIABJREFUQ+g1a8NNQ07FNtabuxlsiqFoxyOfz6NYLLqMNlP9jLS4QZKExK+9vT3MzMy43kf3stN4U0pS95mTPESjzfQdvy4irXYZae+nAd7+fwDSRnQRwix+76/vFWZijo6O8ODBA8zPz0ufNR1WGrh0Oo1YLIZCoeDKDpFE1Wg0sLm56YpkmTKmEQXOhu8cHh5ieXlZju9dZyqVQjQaFUeYjq4WjFleXpa/oWQpcFb+0r3Y1B4nAZUqatzDpqencfPmTSSTSWxsbEiNmVLKjUYD2WxWShlMk/O9mQ1ka1ij0ZDaf61Wwyc+8Qm88cYbUqs+OjrC/Py8aJUzLZ5Op2XvYKCiBWGYKg8Gg9jY2MDm5iaWl5dd5To/eDOVeugRW/JI2uvXKR01rNEeAfwGY3AqFg0fo1MaPS2+AJz1LjPiY3qXxpqGulKpSGqXBJB+iBc6M6HbsszpAAOm6pn+47p17VrfwFoVzdtS1Wq1JML2+9KqStRFB4BKpSLXlKISHFQQDAZRLpclxeZtEWOfJlnvzBywdkpJSX7Nzc3JAxoMBntme3q7F3QbGb/nZzIuWR+LE/g5yQQNWLlcludwY2MD2WxWGMvHx8fieMbjceTzebz44ov49re/La1H+/v7WFlZwdraGorFotR3gZN7PZFIIJ/PY2JiApFIBI1GQ/YDlgRoPNuBPIBMJiO/a4xxDQkhf4OOr07Rlstlqe0z4wScjd4k2S4UCmF5eRnPPfecqLZRLIYGvdFooFgs4qMf/ShmZmbwta99zVUGpJMAALFYDKlUShThyCJ/6aWXkM/nXVkvBgY05sViEY1GQxyrQqGAVquFXC7nyj40m01Eo1FsbW3h4cOHbY027wV+NkyrO46DSCQCx3GQSCREiRKAi/h2FbBGuwd0U/Wid0rmNDXCo9EokskkXnjhBayurmJ1dRVLS0tIJpMuoQIaaK35zbrs5uamGO5yuYyNjQ353UajISlyPeHHD94IOxAIiLADjaEmmT3//POYn5+XWvLy8rIYdRLGOOiDnjMZ2SSdkVFaLpdx//59bG5uYmNjA4VCAeVy+dw1rlar8iDzGLVaDcFgEMViUVjru7u7sg5G1jwPptLorU9MTEhUzuib165WqyGdTovmOq97p64E/a92VOjlM73ItKGuxVpcPeigMcLVOD4+ltQ2a9N0+pgKZ1TM3uRGo4G1tTUcHh4il8thZWVF2qTW19fFkOpIEzgxmIxyaWR12nV6ehr5fF4yQxqBQACFQgHFYhGLi4vniHwcULKzsyMGhmU4OrmNRkO0v5nC1U745OSkaJ0zcqY6GgVZNFmw0WjgU5/6FN7//vcDODHqjx8/xuLiIhzHkWeULZ57e3vS9kVH/PDwEBsbG6KPzpkGx8fHwrgnCZCfIeVgU6mUK4Bhy12z2UQwGMTS0tI58SdG0Y1GA/F4HPV6HTdv3pTzicfjErmzhZUiNQwQrgLWaPvAT9GrEyizyeh0aWlJJndRW5w9nHx4CCoAMZpmOxRJHdlsVggbTJXTwNBI0DD0E9HxwdTtWclkUox2KpWSloxEIiGOiB7NB5ylFPWwEXrdPLdqtYpsNiv1eB1Ve8HxgIFAQM6nUChITzU3P26kHG2qtcp1zbrZbMq50gBzLvjc3BwmJyflWvOL10f/y/tAO3A6ndpsNlGtVl3GX6cr+7mfLPoHa6TAmTOljQsRCoVQLBZ968BHR0fY3t4WY06i2fz8PI6OjrC4uChtRtzEU6kUEokE/uqv/gof+9jH8PDhQ9TrdSGgvf322/I3LD2xpMWIkKlmjb29PdlXvNC90nQMDw4O5HVGj6wHsx5N3gDH/5JDkEwmxdHY29sTmVUAKJVKwq0JBALIZDJizMj0J2nuve99r6zxxo0bct6JREKeW5bBQqGQHFuruDUaDRkaMjk5Kc5voVDA1NSUq0YcDodFArVUKkn/OVn9HErCAEqXIhzHcZH9QqEQ0um0OCLz8/PilFFatVqtiuQrAFe5wS8jomVhR6m29kwbbb15DlN3pOGYm5tDMpnE6uqq1LDj8biM10wkEucewlqtJnXeQqGA9fV12fwPDg4kHaQjaz08gw9+v0xTesg6xcU1plIppNNp+T9TXKwp81ppHXLW2Gmkd3Z2pCZfq9Wws7ODYrGIarXakenearVE5IIp9unpaenzZv2MrWXBYFA89snJSUSjUZeMIUlorHfzQWU0zE2Um9DBwcG5EgK/pwgL7xt69dycdnd35fwDgQAqlco5sZBueJYMulbT6/V3O4FsZTKk6bzRaNOwt+urZUtRtVpFOBxGNBoVFa+1tTWp57J/GDhxAF9//XVxnvP5vBglGm6yyekwAGfqZvy8uR5u9ADaKiN6Ua/X5Vnm35PcR6PGzABwlhlsNBpIJpPSc00SKNu4vMFAJpMRQ0uHlM/P2tqar4YCM1509hn164iY2YZYLIZisSjOB9dTqVSkFY2OON+LkTYNqBZ8YXqcUqvxeFxq/8xA8FmNxWLSPsY9jvui7qGnShy11PnZ+e2/dEbIsh8VrqXR7lU8pZ8NcBgDzghvbm4OqVQKq6urkmqOxWIiZUjDp9+zWq1id3cXuVwOmUwG6+vrkgriTaV7HNmDrAk7gxht1qtYv6ZICdPhjLDZ4qVJc47jiLPAfmiSzMrlMnZ3d/H48WPJDDQaDeRyOaljdzLaPDbr5ExvcQLR9PS01Li5ETBVPjU1hXg8Lj+bmppyCbXw94Ez0lMkEhFjzWupDbU2FNpocxPR14CZCyo1sfXFy9ofN0W0q4SeWAW427FoqOmAdXouNQuc7VNsHyNY721nCNkvzPcOBoPSJ0xexPz8vBj9dDotqepUKoX19XWJVlkSOTo6wqNHj0TpkEaV62PdORQKodlsyv1Jla9uoKPAVDfVwGKxmLR0kTvC1DTlWHnNKP9J54ZGLZPJYHZ2Vlozj46OsLW1JZk4LQnq117FKJOfL40d28lYOohEImIQ+dnQ4d3a2kI4HEalUhFj7jiOfDZMmfNaMFLme5L8S4KfMcbVT8/fPT4+luxoMBhEtVrF0tISdnZ2kEwmcXh4KA4XS5W8t/w6FHh9eU6jTKVfS6MNuGVKvZvisJtfv38fj8exsLAg4zXZi03GNY0KyRCMJqvVKtbX16XWu7Gxgb//+7/H7u6upJD1uEuvbjd/rqOJXsB0EddIw3zr1i1xNm7cuIF4PO5iigNnxk73QtdqNTx+/BiFQgG5XA65XA73798XZSQ+qHzwaRi915mvMcvAG79UKsmNz+hZt4CFw2HhFKRSKWG/63NilJ1Op11GgteXmyzTqsD56I6/w8+CXAJmAR4/fox8Po/Z2VkEg0Hs7OwI23iY++tpBUVRAoGAGBYtfcsaKl/vNb2oN0hdyiEru90GSmIYneWlpSXs7e1hbW1N2NN6VnQoFMJbb72F2dlZvPDCC3jw4IHcS5z1zvYjameT0EQjRkeYDka5XJY6bi8bfbVaFW6JjtCZjub1TafT4uBSUQw4y7iVSiUhcQEQw0onplQqwRgj2QM6IDSE3jWz7sv9Dzib/c1niOfMLhK9p+lhKJQlnp+fR6lUEk4Q0+1M+9NAsiecDhK5Ltxf9LAUGmxKIhtjJLtC3Yhms4lKpSKGmcx1Xm9dmiEmJk5muffS/98vrq3RHhfoWjbbozipK5FICOOUBoDqQEwds42LbNRcLodisShMRe/AC78BGP0gFAoJocy7bmYDKElKkgjbVJiaIquWZC6SzXZ3d0X0JZvNitGmYeT6tXSp3/pp1BlZMWWtv/igeI02CTl0etLptEgosi3OKwKjCWWAe7IavwfgSunpYRF7e3uudJpm1o+qjvU0Q19jbaiBMyM+yHUkWYkggYmfoS4t0el6/vnnXYaHUeXi4iKef/554Vzs7+9je3sbL7zwArLZrIyPpOrewcEB8vk8Dg4O8PDhQ4mEAYhcMSNpyrbq1DiPBZztAdp5B4CHDx9if38f8/Pz0kpFx5bGkil+7kO8TxuNBubn5+E4Dmq1mig3Mk1dq9Wkzs1nutls4t69e7h16xYqlQoSiYScz/z8PJaWllAsFkUelR0bdG5DoRAKhYKoEtJQk4Q2Pz8vLHg6HXQGtAgTldIikYirZEeuCg0uDSdJdty/GKUzzc/0PUmsCwsLAE5kVIPBoIxOLRQKrjS/vs/8SmAX1RZmjXaP0EaEG8jk5KTUvnRLF42ft62LUSSjU7Y+aXY4lbv6JZXptjNdPtApRabC2HKm27lorGl4NNFMz9bmQ1ir1cTxYBrc71xI2vBbs/5XM7L7OW8+QDTawWBQsg9s4dLtKez7paHwKrjRueI19f6MdfyJiQnZ9LWR97bSWYwXtJJVNpuV0sg777yD2dlZkedkayDvpYODA2xsbKBYLEotd3V1VcZHMlKbnZ2VaLRSqeDtt9/GvXv38PGPf1ycRRoO3n80tqzZAifZu0KhICzy6elpZDIZqZE2Gg1sbW3h9u3bODw8FAlRpt31+dLBpNxvs9lELBaTaJX/n56edg00As7GgYbDYTx48ADFYhF3796VCJVz7mdmZlwOEvuwtUb58fGxnD/Ba9loNHD//n2srq66jK6O/hlh08Amk0lXFoX7QCQSQbFYFIM8NTUl5EHuGRRqASAdKLqjgE4F18r9ha9fJazR7gHcwElwIvN0amoK0Wj0HGGL6WQabB2hlstlMXKcg80pXZVKRTz1ftfnNUBcryb7MD1GkhzXTREIeud8INlGxodRa/0Wi0UX0YypQBLSSJxrB79z7Pe8df2bxpRpfD7wlUpFShOUaGX6TafNvPVrZkV0BK6Z4mTsMupiSp8tX4wibCq8d3hTiReVqQiFQtIrDZw9P4x0OaiD6VvWXr/xjW8gFAphfn5ejqVHUNK40BABJ+nrT3/603jppZd6SpOSG0GtdUa/09PTWF5eFtEWPs/ca1guAuDquWbXhLc0xfdi5oHPO0l0fPZJ8iKz/qd/+qexuLh4bt1kfLMFVLdkdRM2Ac6EUbLZLBqNBmKxGOr1Oubm5kQmlZ8dSXB8z9nZWVE65Fp2d3eFwKqFpyh3y/OkEdZZGb4fCYcTExMuTYlu/Ip25zeqts9rbbTbbYi9bJT9yJ6yPYCkLU7qonjKnTt3kE6nsbi4iKWlJennJAqFgkSkjx8/FrGFcrmMd955B9lsVr60t9or6D2yzkbPkS1NNFpedbYbN264sgIkVrRaLRQKBUkf8Ys3PZnSFP/f29vD9va2K0XO3tNO8PIQBjVwlDtl+4hu32J6L5vNIhaLIZvNSjZhenraxYrXcrI64uY10X3oTJdxEAkZ87pEwKin3+yB3/fE09rvTSOkyU267tuOocvX9HXR0ZcfNjY2sLi4iGg0KgzlVCqFfD6PmZmZcypb1NwOBAJ4+eWXsbGxIQQ6RmwHBwdYXl6WCI9O6/r6Oj72sY/1XNdkrzhlPelchkIhmXTGOjWlSfkzTYxaWloShrPuTadzQZY4S1YsQfE5oqELhUKSOVtbW5OMghcki/K4ZG/3SpAlyezmzZvSxkaH6ejoCDMzMy5Ne10D39zclICEhFWy1AFIGSudTkvPNw09HXHW23d3dyU4yWQy0lobCAQka0BeTa+fJ3CSgRmEf+SHsTHa/TJqe2WQ9/IevYBtXdFoFAsLC65hGisrK8JS1GPt6KnqujUJWzRs3Og59WoQ8CYiS9pLyqKqEFPgrGOvrKzI/8PhsBB1yAZnGp8bkBYt8bLai8Wi1LdJYumGQT7DduDaDg8PXelqOh9M6bPmxbo+2/Do2DDFro02HQpG0xSkyGazUiao1WrI5XKSRSHLv9/+eeI6p9fbyTz6zTimk8WUo56exFQwnUVtuB3HkXvWuxFqzWc/PHjwQEhh6XRaWpY4JCeXy8n7MQOTz+fx8ssv43vf+55s+pubm8jlcnJP8PnlGl9//XV85jOf8Z3e54fj42Nks1msra2hXq9LWndyclLKOlpshYNyDg4OJPrf399HPB7H0tISDg4OsLu7C2OMqKYxJcx7ntyZ4+NjTE5OumSReR2j0SjeffddvO9972t7TXWZiscfZEBGIBCQz+D4+FjqzMYYYcUzs8fSBMsHvM6sPfN+o0hMPp93CatwyhsDETrgzKIdHR3J/ceInfeg33AnPzDrUqvVxJl4aoz2MOi2KQ67AXITYi/20tKStEdp40ejSFYsvXSOx8tms3jy5ImQtOr1umz0ut2kH9CLprc9MzMj7SnpdFoMcjgcRiKRcMmTUkCARoxePFNPOnVfrVZF+Ys1Mk3kIEuWc7J7NVTeVqhBo20tpsKNyHEcEXmp1WqYmZlBpVIRg03PnNeM149Gm5s2rwmFU3ieOzs7Uh6o1+uiZseHnw/6oA/pdTXcTJF64acfzv/zZ+0ibUaFFA+hIfdG2t3YunSi9QZMNS2Smbhx657narUqspaMPtkZsLGxgf39faTTaVECDIVCuH37tqunuxv4/NfrdXF6dNo+EDhRMdzb2xPHm4HE4uKi9Guz9XF7e9vFlC6Xy/IscL/SqXEa8Vgs5uqiODo6kpp3O1AhjM59KBTyHaXaD5LJpNTN6Uyw7s9MBwmxwEkpgiVL9sbrMb1s9WJHCmWNSZ5jpkEPQWHErstwJPd1Au9LXl/2s2undFBce6PtF61pb6od+kmPsz7KevDCwgKWlpaQTqeFLc6Nn+k13RrFKJssU6qD7e3tiQwpo7h+wVYmSovSQEejURliwPR3Op2Wn0UiESGdUOmMoi7NZhOlUklS9pQdZURDchpJLExbadGXq6jlauIQ10rJRvZus9bFUgHbd+i40OliilwbbW+b2+bmpstoc8Nm5MW0fb/lmutqrIcBI25urHyN86V1L6x309Mb4cTERMf7jw4o72MSqVKpFCYmTgaBsJTCz5GDOaampkTsiJEknddcLidGkW1Nd+7c6SvS3NjYEAUvngfPmcam1WqJrsLCwoKQyCiswrWxNk92eLPZlKwZz4WOON+HGTWy2RmV02nolBKmg6UJrMzK8TkE2pcsvKDTwhIEuUS8N/h8MWDguvUeykib0XQ+n5daObkCzEzy2rB2TZEXOoh0xNiu10udnrKydDaoWjcske1aG20+mBdtIKLRKBKJBBYWFrC2toYXXngBN27cwMLCgkSvXrYx61ocDr+5uYn19XXcv39f1MF0C9Eg5xAKhUTpKBwOS52O67179y6SyaQosy0uLroia50GBiDKPZVKBevr63j06BGePHmCnZ0dFAoF13vr1jMaNt3S1cv5+AmO+DGvO/Xga7a8rs1R1IT9pZoRro322tqaq0SgFdV0SxjbgtjmVq1W8eTJE9fwFkYB+jr0+9l2ug5PO5iO9UbNnTZ6rV/Av4lEIq451ho7OzsIh8Oo1+tYXl6WWjB766mtwM2aa+K9w5YjGkg6AZlMxqXjzXaxXkEdA0an2jDs7+/LOF9dq6XMKQmk5XJZNCE4GYv9znyuabS16ApVv/h7jEx5nYwxPZ2LzpJMT0/DcRzXhDI6YPxcvYZcO2PJZNJFEKMTw3IBneNWq4VisYhYLCbsdDLIgZPnSPfma20FpvRrtZpkGzjrIJ/Py17K9DodcGZfOkHfl2Sts3vnmTbao0a7aMc79Yq1bNa1vcpFbBPg5s1aKx8GHYV538+7Uftt3HygmQqi0fa2cul18osELK8yGz1uRv16JGipVHKxJztdv4twoHoxXv04cJQs1X2g/JxarVbPRrtcLksdf29vzzLFh4CXNT4omDbXJadKpQIAuHfvHrLZLF5++WXpt2YrUCQSwd27d8UYk6fBqJeREtnKxWIRmUxGCEscF0mHcXZ2Fm+88YYY/WazKSNlGSE2Gg1Js964cQPpdBqZTMbVw83ojs8r25WYPWPHBI03jVUikcDx8TG2trYQCASwvr4O4IS0Salk3WmiRW2y2ayw5Hl9YrFYx8+FP+O/vGZatpR1YF3SI3mN58ZjcEgJ/6WSIqN3ijaR1X7nzh1pZ+Xx2TFCVj8dQBL76Awwy8NM3ezsLLa3t4UcR3lkliBZ/+/1fqSB1yqWw8Aa7VN4W6O0N0iDp8VH6J37SQ1qI6NZ0bq1SKtB6Shdp2W5Lq+oCG9IetSMnkmEY6St18xWND37mg8BBSb0GFCmfGmoesFVpcT7xdHRkQxGYCShswXtjDbTh/q6MGVuDfZ4QKeIuUknk0nUajW8//3vx3e+8x1J9bLFiT3GR0dH+NKXvuTSFuDsADq13HTZ00z9bm7EkUgER0dHCAaDSKVS4iQDEJUuXVYi74HiK1rkAzhTcSMZi38bjUaxt7cnM743NzcRi8VQKpVc5SpmCcLhMOLxuJC4fuqnfqrrdSS0sW7H1u/H0fIaMn0Mkte8GZZOxK/9/X08fPgQc3NzmJmZQTKZFLlTCitpKVVG8K1WS64jy4NMaf/Yj/2YdAz0Cn1tmEXV+8Ko1NGs0T4F+5p16xQf7vn5eaTTaUk1c3h7L94W0yIUVmANmZOpdP2Uxlg7ENqA8DhUHJqcnEQ8HpdaejqddrHDqX2uhVM0SBpjuiibzSKfz0uLGn8+rNh9P/yBfjDMcQ4PD6VNRffpMtLWZQMabbZ/0HAzXTfoOryZHe9xrCMwGJie5vPFaJlzo5mV4vPAaPbNN988JwakSx2aoaw1r7lZ0zmg4WY7oR6uQQPSaDRkVOXW1pa0YbL+q9PMVAqjo8FU7Ysvvojvfe972Nvbw+3bt7GzsyNRM7MAJNEx8p2YmEAul+vpGnrB54Na4QR5I35/18lQacfYGOMy2P1genoauVxOrvfBwYHUwmlAed0BSNaE9X4qtbEcsL9/Mgu9X4NNkhrfMxwOnwt4dAlhUFwboz3qjd/795OTk0JEWlpackXTq6urIk3KOjZrYH7QdUzWMWKxGNLpNG7cuCETbKimxRuVNSoaeO8EK66R5DOSTUimYH8400QLCwtCTNMZAYoHPHz4UMZlsoe8VCqhUCjI3OtB29CIXmqzgxirURg01vkoy8qWL2+PNjduPSubZYRBU11+18WboXka4cf2BgZPiXuNAlOp7NvVvxOPx3Hz5k2X4hcVyfh3Xuh0KjXFdUtjIpGQc2BqVZegDg4OZCPXg0HS6bQYc7aLsreZbWw0uhQFYm358PAQd+7cwTe/+U0cHBzg5s2bmJ+fx7vvvis1dxpWRvckaBlj8OKLLw50rQFIx4XXaJOpTafDK8fqZ6i0cI6ezDcIbt26JVlLRrh0Chila/EbOuysO7MrgWI0/a6DETqvDdvV6DgwtT4KjI3RbrdJtdvwu9VP/f7Oj/hEMMqdmZlxqZtFo1GsrKy4pl7pFLMfeIN6iU/RaFTUjNhsr9Pi7AVnxD85OYm5uTmJzkmw0EabI/+YIaDhoaPglxFgS4Nmh+/u7mJjY0PkVavVas/91p0+g1GTqUZpzLgx8l6iR65LEvw9/XNyFPr1mJ8VYlm7li+OatR91bzWXs1xDaaPNTOYBCVuurxP9/f3sbW1hVu3bslmTOeXGSqKqei1kozkXW+r1cLGxoaLzU1jzFaxw8ND3Lx5EwsLC7JJ8/wSiYSQIvlMMiXfarUkbZ1IJETxjJs+a61aZpSZtO9+97vY39/HBz7wAUxMTODtt99Go9GQmjjbxphtIBkrFArh+7//+/v+TNmDTYOs7322eum0Nn9OXXBtoHl+usWP/x8UJOklEgmUy2XhELBdDDhr3aWWA0fzVqtVIYtxb+4F7Ndm7Z33Gz87lmmo3Mg16GsxCMbGaF81eDPOzMy4JD5Z06LMZywWkxS2X98pNyQymgOBgEyi4ehO9hdSUtBrtLU6l05/sydTp+4ZmfN9aOyZBvRbH8k5mUwGW1tbkhbf2tqS9C8HBgyTyunkcHEtfr/nxxC/KJAMyM+SRtyPl8AUGOuS/IxHIZ6ijzPMcccB7cRVuDl7XwPQcSPTRpDQ3BPdasTPrlqtYm1tDcDJzOVAICBtjvpZYeqYTGe2NtL4UHdfq3LRSYjFYqhWq1hYWHCRnbSTzGdVi73Q0aZxoKY231fLa1KQI5lMCont3r17aDQaiMfjePnll/HVr37VJeJENjgJdlorIJFI9EQqJfh5cVynX9TMPmTdSqf7zL0iK1oYh07FMNB95sDZvcRrx1YxkgvZZ8/Phc4RSym9Gm3NEGekTWfecRxh8BtjJCvL/w+DZ9ZoezdEGuxoNIqlpSUsLy9jcXERqVQKKysrUsdm1MtebODsJtHtPnydKkbGGHmYKSXIjV/Xr6nVS0JKKpWS/ms979WrK07QUfB7ECg2srOzg42NDdy7dw/r6+vY2NhAJpNBNpuVDVLLBHqNbDt4GfDtbk6vse523GGMlzbAft0BfNB66ZPWpEJ+znS6vG1pvRjdTg+vfi+LztCGhFyM1dVV1whMzoRn61IsFpMRuKx3GmOQTqdFCYuZFUaWwWAQpVIJwFmkz+OQAMoUuHbGNXOaxorZOooazc3NiWYDs2N6ytfi4iLu37+PcrksWYzbt2/j8ePHQiSlEdLdEWR/MwIMBAJCfuN5+IE/19Gj7kHXoHNDjQpeG8DNFNfH9htZ2wvogGghKMdxcOvWLeGZTE5OSnBCst/R0ZFoMHiJwBSP0e16ek/wrrNdvV9fKxp0rRUBQAiSw+CZNdpe8CHTZDF6qiRxMYpl7QQ4v5F72eLcNFjfYKSs2YVeo833YZ2aCl78l5G1NxrUvdN+Gz4FQPRIUGpls7bd7tpcFJnsIqFZ9/y/hs6K8P+9HreX1ywuHn6tRkxHBwIBed7o6C4vLyObzQI4i4KBk003l8uhXq9LJo2pVR774OBAWMlMj+/v72NhYUEMOyNtRpcksPFvIpGIzNYOBoMyzpLiTSSXcpPf399HJBJBJpOR55Nr/shHPoJXX31VygT8uXZG5ubmJHvGejlnynvV57zhXCY8AAAbsklEQVTGJBwOY39/X/YurX3uBV/rpZw2SO1aOxEMnrSB5Pq5Rp6TFkahEWeZhHrjVK1kypwa8vre4lS2ThwfP80A7zkPIu3qhTXap9AtVawp84v1Y/5Lg6lFO7zH0rVqMhkdxxGBf7Z8MCXGFIpOjzP6ZyqPNw7gfkiYutUpW5JZ9BelNnd3d1EqlaR2zfauduAxnyZ4syL9/q0XT9v1uc5gFMdaI9XNdBcE2/SYUaJxajab0gvNlj5u7jTg2nBxuAX3h0ajgYWFBWSzWVE30xs1x4G2Wi3EYjHkcjkxBowQGR0yZd5qtbC8vIxoNCrR/+HhIZ48eSKtntVqFYlEQgzUq6++ik9/+tPnDAQjZkZ8dADYBkVoA+TNOFwGvPK/jJRpvKkudnx8LFktkgkPDw/l/OgM+YGZT12Dn5iYcA2D4Wv8PPxKor3CL+swCKzRVtCRmTbKmuGtW7QAnDPaOm3KY5I1ODEx4RqRp9Or2lNkVKBrb9prJHSfJ+s6PCZvaG20a7Uadnd3kcvlUCgUZCjIoBKq44Reo19vVmRYPI0OzbDwEtH4rAy7WfUK3XIDnPUCc8Qk09pa3CiRSEhPN18PhUKiUU32b6lUcnEfmIFjip3/T6VSrr2C0Tuj/5mZGUlr0ynY3t4Wo0F2M7saWq0W8vm8qxyWzWZdEpnNZlPquNls1te4ePevbtDaEZ1at9p1UfjxGLr1K7NG7jXarJvr9fB68XiO4/SdgtaRPzM1nKeu1+A9F69j4T0vTQSksR+WhAaMkdHuxvYeBP1sqJpsRMPKLxpFrkdHvF4Yc6JmxGPpukmz2RSRfkbZui6tH27NWGaPMFWcuD7dsnRwcCDqQbrvWBvtvb09MdyFQgFbW1vS1jWKunKvtdx+a739/H67770p8FHWjHvtfPBem3atbteh9NAJXgYu04LDHlP3L/fyu4yeuQYAroix2WxKSyYNsh4pOz09jXQ6jWKxKMxxCnLQgFCUIxqNSh2Z0pdM39JZIXksl8uJU82Ws729PVcfNOuzPHapVBIGOMlzsVgM+XxeGNFTU1PihD/33HMdr083A6IDCWYBOrVu6XYmbcRJOvOmiruNINbKYXTCGNQwbc97ioz+3d3drp093cC/pSOl92/vuWhiKj9jfY2YkufvDKt3QYyN0fbDMH2r/W6A9Kr4MOs0FFNOjH57XXswGBTjEAwGhQhCg61Zqjo1T4Oro3L2BnN2L9WcONeaDzijBGOM630AiEJSrVYTCU6vnGov17Tb7/RKMOsVne6DXslv44jrbpzbwRtpc9OlQ+rX9tIpWtMRDDfETr9PshUjYaZNo9GojGxkHZhtaKVS6Vy2aX9/H/l8XtqyOAGLqXa2Z05NTbk6RSYmJlwDSMhK9/aCaw1sTYCiTCfr0qHQyVjJfD7v0oZge1cul5PMHXvFV1ZW2n4+xhiRD+V19atF12o1zM7OigPkFVXRn48WpfH7vPTxeT94e/R13Zr/6haqmZkZl7xpPB4HACEN9rM/d4O3Bq5T6FwnOwg010Df2/xMqZU+qrWNjdG+iEi7H+hIm8aaDzfVj2hg+6lrsP1F/502pNpo88EHzm5cpr31zGY9bSqfzwtjlP2J7UhyPC+Os+NUqsuqU10U+s2mXAT8sjrd2OFPK7wtX6zjaYazN1XeSaQmEAicU5bqZLD1Rs8eXJaSKIpDQimNIuuWXhwdHQljfHp62jWiklktnptWRjs6OhIdbJLhaLgnJk7GZLKXmsM5GJVVq1VMT0/LWFng7H4hEYtOOg0C6+KpVAr5fF7a3fzAyFhHhd79jPsS12aMkfq/3/F0ZkWnkf1kT3kfeFvhWq2W73H4WZfLZcl28F8K6eh+7F6g1+WXcSBpUPfN675sXiPNidDXju9BHtQwehdejI3Rvmow0qa3S2NNA8l0Ej2+frwmHV14U6T0zrygIWcNjtFxNpsV7WsKpFAHu1QquW4ObypWZxFovHuV4ezHyHhburodq5ux6+W9OzHE+TVM5kYfW5cBvP3VfmvoFU+LIfdG2oD/LO2LAKNC4KymqDd3plYpYzs7OwvHcXD79m289tprHR3Yw8NDlEolV4qU0RQdk3Q6Lfrg29vbaDabmJ+fl1GQrVZLon0acZLctra2EIvFsL+/Lx0eunecDsbR0ZHIoLKtzRgjoynn5+e79mL3U65gdEtlN79rriNtMs0J7wzpbulxliBYuuC9pNnX5CVoR7BbvVg7EnSqtNFmf386nZZhM7QJdBK0E8Fz59q8WSQ6jaOMsoEejLYx5gaAPwSwCMAB8AXHcX7HGJME8L8APAfgEYCfcRxn15zsWL8D4BMAGgB+wXGcb41sxRcEpmz29/dl3izZm5w1y9YvfoiaVa6Po2vJnaA3fu9amMpmO1ahUEChUMA777wjrO9KpYLt7W0ZDbm7u9tzqhu4eCMxbkaon0hbOwEX1e42btfnaQA5JTRuZHxz86eDq9O+x8fHKJVKuHv3rmSu+IyzvYha/zT+zL6xltxqtTA1NYVsNivkNeAsYmUKnfVnGsJmsyliR3zmK5WKGMJqtSppeRom3pPFYhHNZlNkTglvjdmLfklahFe+VL9fO5Khn6HrBV7muj4e34/MfmZKupHbtAjKzMyMy8lmJpViOt5Wwl7AbIcmtQ3S3tYNvUTaTQC/4jjOt4wxUQDfNMb8NYBfAPCK4zi/aYz5PIDPA/i3AH4CwPedfv0jAL97+u9YQ0eifHiYvqpUKi4yARnhXqPNOrLu/fXCS5TSdSX+DYlnlBT1fnHsINu19DxnC4tOeBaccGavGNXSoE5PT8MYIxEayVysf8diMbznPe+RqV9MixeLRYlu+bzOzc1JfZkZMfaBUzCFxKlEIoHHjx8jkUhInbjVamFvb0/WrCeN0TBtb2+jVCrJuE0v6FRQPIXTCAH4Th8cBoMYsYsES4m6174TvD3nOkJm5M6szKDlwlG1dHVDV6PtOM42gO3T76vGmHsAVgF8EsA/Pv21PwDwFZwY7U8C+EPnxAK9aoyJG2OWT48zcvSagu0GnTpuNBquoQKsl1AzmDq8uo+b3q+3jtxujYywtQQjX2cvZa1WQz6fFxEU3Vutx0OSiGYxHMaNxHZBPI+xdML92nwI3dbTzWgwYiJBTL/WbDZFtQvAuXJRJpORKJkkMKauyZ6ORqOumQEAXMxlspypOx0MBlEsFoVHwh7pqakp11qSyaTsH7Ozs8hms/jJn/zJsTGSlw0/g6z1K/zUIAkGWDqdT7Ia69CjaL/qtAbvehjItSsx9IO+atrGmOcAfADA3wFYVIZ4ByeeO3Bi0J+oP9s4fW3kRrvTJtsvg5kPMFNXfFj4gFL7mwQOLbTC1Fe3D89r0JmW8TPajPaphFQqlcRoa4NNJusoiQ7XEe0+Zz8nyoteerxHkcruNzV/EbgMJ9yPiKZThd7nhJuat81HG0VGr91gjEEkEnGxuVutk6l2jLKYZqZ61v7+PlqtFur1uhhU9koXi0WJzJiSBiDPPKU9ObaVPzs8PES1WhXZVH1ufHY5g2B2dlayelzPKJnQVwUaNG/k2Yn5z79jyxtwRjpkShw4EUap1Wque0nXrFnSpIHUw2omJk7GtPI5473QSxuc/kxYwujF+NMhbUfm6wc9G21jTATAnwL4ZcdxKh5Sj2OM6WtXM8Z8FsBn1THO/U6nTa5bi483yu22YbIHGgB2dnZQLpdFjWxra0uMNDcQPeeabR78v7ePm14WPUCdPm8nzkLvf3d3V2o39Xpdekb5xb7ti8CwxmMQQ9fvfdDt570Y7G7n2a6fWh+j13X1ck/3e8xBcVFOeKvVcrG9uXGT+OiHSCQitWRNENKEKr3RtzNoExMTQhQjmZSGmMdktozpb6alWQPnXGy29LDWCUDOg3VtAFJ3JlmM0Tkjurm5ORE7oVEGzrILdOBJjqpWq5ifn293eQeCH4u7H3jbs3oBW8u8RroXIzk7O3uOiKYzn5VKxTfzwpo1BVJ4vl5jqf+WmRGdgdHrPT4+lqyNV/qVgZtf77u3hY3HHhY9GW1jTAgnBvuPHMf5s9OXM/S4jTHLALKnr28CuKH+fO30NRccx/kCgC+cHn/onWhYZrCuZzOVwjoHmZr8YrsIDS4b+vnlTXlrAXktdM91+xHRSJxgLzW/arWai6Axzu1al5FuHpQV3kt0DUAU5i6aNHbZqfmLdMLj8fi5SJuRpHeTpSPbbDbPzSTm5s7UdDgcRigU6sh8npiYkMiYrVGMrrkWMo7JKicru16vY25uTrQP2Kalp/rxOCSMsVbKc+Fx6FADkBncvMY8N+4xeu430+qjrkkD51nc/YAOVC/7Da9Duxqv10jq9wgEAiIHS6dHOxwkDur2WL6n3he9kbY+frt+aoIlCkKn2CmOpc+FmVm/PZ8R/uHh4VDXX6MX9rgB8HsA7jmO81vqR18E8PMAfvP03z9Xr/+SMeaPcVL7KvdSz+5nU9SGrtvm22963HEceegYTddqNVfLF1PaXAeNtjbq+gOkXCE/PG+07ZeKpeGm+AlFVnjTelXanib0e05+DPxej9dLpN3ueP1G6L1ikDX1efwLdcJXV1cdb6TSzmDPzs66GMB+4GbLOcid2scYnfG5peIYcXR0JH3QNMCMmMrlMiqVCmZmZrC3t4e5uTkx6DTyNDZs1+LULaqbZTIZTE9PS/ZLvwe1GqampiQDoA0aGeocgTlKDMri1tBdMp2gzykYDEr5gdAKYhokhHn5Dfq+YO+7t2+fbPl2PdF04shZ4qRG4KxFUfdhe50T3gNeXYt29zZ7zuk00n5citEG8CMAfg7AG8aYb5++9qs4MdZ/Yoz5RQDrAH7m9Gd/gROm6QOcsE3/xdCr7AGjiLT5L+se9Ma0+In3Czh52GjgQ6GQpMsZodFok+DSzeDydT5ojMz1/1mr1RH702jAe8Uozr3dPdRPq9i44zKccG9Nux2897Vuy+FGxyimV2NDo83nS5eOSNjUKVcAKJVKCAaDqFarogkeDAal9UqTPDmMIhQKoVAoiCGr1WpotVqIx+Oo1WpyvzBVOzc3B+DkXtrf35dIm3roPI4WZfGSS/2MCXBWO/ZGj+MARsT9tpm1ay8D2muXa8eEZRIaSd0XTqPPe1TX3TsZ1kHOhevycjaGQS/s8f8LoJ3b/098ft8B8Lkh13XpYM0ZwEA1YpLKqM6jjTa9OBrti4I3Ous1+nxaDJJFTxhrJ9wbEQ5S/mG605z2zVIkiZkqavXTcDILxgh8cXER0WgUc3NzaDabkuLksWu1mozbzOfz2N/fF6VCb6qVDv/k5KQcLxwOY3Z2VkQ86ARwJCdb07zTqaanp30NinZu+P11x7D9zZ3uIzqVlzXAZtSwimgjgjbOOlLn60y/W1hcJZ4FJ9zLLwFwLuoGzkeux8fHMr62VCpheXlZIjYdIR0eHmJyclIM9cLCAm7evIloNNrRUdYRcbv/RyIRKaFRhUvDL1ILhUKiYQ5cnvqcxdVg7I12r4ShcYkcmQbRfduslVt0x1Wk+QdhrF80rvr9LwJ+keLU1BTK5TJarZaUlnQNlGQt9kD3A2+01ktNNplM4saNkxJ+vV5HJpNBNBqVdXMaWKvVwssvv9zXevoBHQ5NNu1Uxx+25/giwNp0vV537dnd1MsGBY87NTXlIuwCZy135BJcZoss0+/e2v6gGHuj3Q2dyFxXBabmvGIrF41BmdTjhsv8LLuR2C76/du937gJvQwD3frC1immL0OhECKRiBBAdcsXU81kj7dLZ/oNfxiFUQiHw1hZWXERp46PjxGPxy+E3e0Fa+xEp9GY4wbd3qpFZIDz/c2jNODs/KECHj83dgPws2xHViNZUP9cE9a0xCnPk+h0Ho1G4/LY41eFTvVZv2isXaTdq6DGKDdmGuqrjhoHZVT38zuDoFP2xO89e3FE+j1X7ex1+nkvPeDt1tPPmryf23V1uLzQrS+Merh5sebIjZCbnm750ipYutXSCz3Zi8MrRmEMuMGTncwxvZeRfva2j5I574WXNOt3fS4quiUp1ruucDgM4GwwidcIkuHfrv7ejnTHv/eeC1vF+Nyw9Y7H0G25FMfyAx1Hrp/v12q1EA6HzzlOzqnKmZfNrtfIfzt1SPSDsTXaRDcRFaKbAMZlQ6/7qtZy1ddgEHQrfYwaOqodl+v1tEXaNNgkYpIN7jgO4vG4S0BF99CyN5htV+3EWbysYQq1jALccPXksMusF2sj0WmedbVa7arK2C5SHwZeA6dfb6cApp2PdvX3TlEpW8M0dJYGwDlxFUbgnVLjeh1ecSD+nTdroM/Rex683o7jnBuFOgzG3mh3Qy8RUS/QG/ewUfhlbrhPU1R2VRjH6zeOaxoEuq2LxCqd5tYbHuuNuh2Hz5Jui+r2fvzdarXqqkd7jRs32U5pWhoCHiMQCIgO+iDopEymnRivYheAjp0n3a6N91ijRLs2KZ2h0GCtvhP67Snn9WJLrTHGNS6U9xGnvvVyLC/4mfWS4vZe71EOc7r2RnvU8Etr+kVig6ZNB12ThcXTDqaDh23H0SlRTWCbmJhAKpUSg81orVublFYCG0V6s10UqXUYLPt7cAzbLnYRGKXTZI22hYXFU4lOghbeaK+doRx1hNpNmWwcSWUW44VrY7SflnShhcWzDC+JaFzblSwsxhVjb7S7tXRdhDG3DsLlw17zpx96mpau7bKuzT7aURlxTvryqx97W3csBkM7VnqnnuTreO1JjByHAU1ja7S9xnrUm3q3dp1+/+Yi8bQZtG5tXb38/iC/c5kYt/VcFfwIX7r1hXXsYdLQ7QxHIBDwbUfy/s1lRvvdWq+uY+bBjxXdiaXNa9DpXLtdo1ESu3pBrVZry94HeiOnjQpja7S7wW6KTxd6nZxlcf3ATZ1M4nYziYfZ+PwMR7t2JOBs1jPXcJno1Ppz3Qx2O2dLy8d6z5XXftD2tIvqOe8EDhkZh89t7I32VRhn6xBcHey1f7qgN3U/cYlee4x7fQ/vsdsJWgzrJAyKi2y9Gid41e00ern248YAH6fPbeyNtoWFxdMBryAG0euc5n7hJ8JBXHZ69VmD7pv2wl774XC9cjEWFhYWFhbPMKzRtrCwsLCwuCawRtvCwsLCwuKawBptCwsLCwuLawJrtC0sLCwsLK4JrNG2sLCwsLC4JrBG28LCwsLC4prAGm0LCwsLC4trAmu0LSwsLCwsrgms0bawsLCwsLgmsEbbwsLCwsLimsAabQsLCwsLi2sCa7QtLCwsLCyuCazRtrCwsLCwuCawRtvCwsLCwuKawBptCwsLCwuLawJrtC0sLCwsLK4JuhptY8wNY8yXjTH/YIx50xjzr09f/3VjzKYx5tunX59Qf/PvjDEPjDH3jTEfv8gTsLCwsLCweFbQS6TdBPArjuO8F8BHAHzOGPPe05/9tuM4P3j69RcAcPqznwXwEoAfB/BfjTGBC1i7hYVFH7AOuIXF9Uew2y84jrMNYPv0+6ox5h6A1Q5/8kkAf+w4zgGAd40xDwB8GMDXR7BeCwuLwUEH/FvGmCiAbxpj/vr0Z7/tOM5/1L/sccBXAPyNMeY9juMcX+qqLSwsBH3VtI0xzwH4AIC/O33pl4wx3zHG/L4xJnH62iqAJ+rPNuBj5I0xnzXGvGaMea3vVVtYWPQNx3G2Hcf51un3VQA9O+CO47wLgA64hYXFFaFno22MiQD4UwC/7DhOBcDvArgD4AdxEon/p37e2HGcLziO8yHHcT7Uz99ZWFgMj1E64BYWFpeHnoy2MSaEE4P9R47j/BkAOI6TcRzn2HGcFoD/hjMPfBPADfXna6evWVhYjAFG7YCfHlMyZ/V6faTrtbCwOEMv7HED4PcA3HMc57fU68vq1z4F4Lun338RwM8aY6aMMbcBfB+Ab4xuyRYWFoPiohxwnTkLh8MXdwIWFs84uhLRAPwIgJ8D8IYx5tunr/0qgM8YY34QgAPgEYB/BQCO47xpjPkTAP+AE+LL5yxxxcLi6tHJAT8lnALnHfD/YYz5LZwQ0awDbmFxxTCO41z1GmCMyQGoA8hf9VraII3xXRsw3usb57UB1299txzHmR/kQMaYjwL4PwDeANA6fflXAXwGJ6lxccBpxI0xvwbgX+LEAf9lx3H+sof3qQK4P8gaLwnX7TMfN9j1DQe9vr6f57Ew2gBgjHltXElp47w2YLzXN85rA+z6LgLjvma7vuFg1zcchl2flTG1sLCwsLC4JrBG28LCwsLC4ppgnIz2F656AR0wzmsDxnt947w2wK7vIjDua7brGw52fcNhqPWNTU3bwsLCwsLCojPGKdK2sLCwsLCw6IArN9rGmB8/nSD0wBjz+ateDwAYYx4ZY944nXj02ulrSWPMXxtj3j79N9HtOCNcz+8bY7LGmO+q13zXY07wn0+v53eMMR+8grWNxdSoDlOtxuXaPXVTt+zz3NN6xvZ57rC+sbgn7TMNwHGcK/sCEADwEMDzACYBvA7gvVe5ptN1PQKQ9rz2HwB8/vT7zwP495e4nh8F8EEA3+22HgCfAPCXAAxORqn+3RWs7dcB/Buf333v6Wc8BeD26WcfuMC1LQP44On3UQBvna5hXK5du/WNxfUb4Hzs89zbesb2ee6wvrG4J+0z7Vx5pP1hAA8cx3nHcZxDAH+Mk8lC44hPAviD0+//AMA/u6w3dhznqwCKPa7nkwD+0DnBqwDixi05exlra4dLnRrltJ9qNS7X7mmbumWf5x4wzs9zh/W1g32me1tfO/R9/a7aaI/rFCEHwJeMMd80xnz29LVF50zqcQfA4tUsTdBuPeNyTcdqapRxT7Uau2tnno6pW+O6Pvs8jwZjdU8+q8/0VRvtccVHHcf5IICfAPA5Y8yP6h86J3mNsaHdj9t6MIKpUaOEOT/VSjAO185nfWN1/Z4C2Od5eIzVPfksP9NXbbTHcoyn4zibp/9mAfxvnKQrMkyrnP6bvboVAh3Wc+XX1Bmjsa3GZ6oVxuja+a1vnK5fnxjL9dnneXiM0z35rD/TV220/x+A7zPG3DbGTAL4WZxMFroyGGPCxpgovwfwT3Ey9eiLAH7+9Nd+HsCfX80KBe3W80UA//yUNfkRAGWVNroUmDEZ22qM/1QrjMm1a7e+cbl+A8A+z4NjLO7JdhiXe9I+07ha9rhzxu57CyesuV8bg/U8jxM23+sA3uSaAKQAvALgbQB/AyB5iWv6nzhJqRzhpObxi+3WgxOW5H85vZ5vAPjQFaztv5++93dOb8pl9fu/drq2+wB+4oLX9lGcpMm+A+Dbp1+fGKNr1259Y3H9Bjwn+zx3X9PYPs8d1jcW96R9ph2riGZhYWFhYXFdcNXpcQsLCwsLC4seYY22hYWFhYXFNYE12hYWFhYWFtcE1mhbWFhYWFhcE1ijbWFhYWFhcU1gjbaFhYWFhcU1gTXaFhYWFhYW1wTWaFtYWFhYWFwT/H/T4W5RJYFTCgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "from PIL import Image\n",
    "import numpy as np\n",
    "\n",
    "# 读取图片\n",
    "img = Image.open('./number.jpg').resize((256,256))\n",
    "\n",
    "# 设置卷积核参数\n",
    "w = np.array([[-1,-1,-1], [-1,8,-1], [-1,-1,-1]], dtype='float32')\n",
    "# 创建卷积算子，卷积核大小为3x3，并使用上面的设置好的数值作为卷积核权重的初始化参数\n",
    "conv = Conv2D(kernel_size=3, stride=1, padding=0, \n",
    "                weight_attr=paddle.ParamAttr(initializer=nn.initializer.Assign(value=w)))\n",
    "\n",
    "# 将读入的图片转化为float32类型的numpy.ndarray\n",
    "inputs = np.array(img).astype('float32')\n",
    "print(\"bf to_tensor, inputs:\",inputs)\n",
    "# 将图片转为Tensor\n",
    "inputs = paddle.to_tensor(inputs)\n",
    "print(\"bf unsqueeze, inputs:\",inputs)\n",
    "inputs = paddle.unsqueeze(inputs, axis=0)\n",
    "print(\"af unsqueeze, inputs:\",inputs)\n",
    "outputs = conv(inputs)\n",
    "outputs = outputs.numpy()\n",
    "# 可视化结果\n",
    "plt.figure(figsize=(8, 4))\n",
    "f = plt.subplot(121)\n",
    "f.set_title('input image', fontsize=15)\n",
    "plt.imshow(img)\n",
    "f = plt.subplot(122)\n",
    "f.set_title('output feature map', fontsize=15)\n",
    "plt.imshow(outputs.squeeze(), cmap='gray')\n",
    "plt.savefig('conv-vis.pdf')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，使用拉普拉斯算子，目标的边缘可以成功被检测出来。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 5.2 卷积神经网络的基础算子\n",
    "\n",
    "卷积神经网络是目前计算机视觉中使用最普遍的模型结构，如**图5.8** 所示，由$M$个卷积层和$b$个汇聚层组合作用在输入图片上，在网络的最后通常会加入$K$个全连接层。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/c08f9a5599ae4f9d8eedb0131c54af859fcfaae003224bc6b247d60391b2b102\" width = \"1000\"></center>\n",
    "<center><br>图5.8：卷积神经网络经典结构</br></center>\n",
    "\n",
    "从上图可以看出，卷积网络是由多个基础的算子组合而成。下面我们先实现卷积网络的两个基础算子：卷积层算子和汇聚层算子。\n",
    "\n",
    "###  5.2.1 卷积算子\n",
    "\n",
    "卷积层是指用卷积操作来实现神经网络中一层。为了提取不同种类的特征，通常会使用多个卷积核一起进行特征提取。\n",
    "\n",
    "####  5.2.1.1 多通道卷积\n",
    "\n",
    "在前面介绍的二维卷积运算中，卷积的输入数据是二维矩阵。但实际应用中，一幅大小为$M\\times N$的图片中的每个像素的特征表示不仅仅只有灰度值的标量，通常有多个特征，可以表示为$D$维的向量，比如RGB三个通道的特征向量。因此，图像上的卷积操作的输入数据通常是一个三维张量，分别对应了图片的高度$M$、宽度$N$和深度$D$，其中深度$D$通常也被称为**输入通道数**$D$。如果输入如果是灰度图像，则输入通道数为1；如果输入是彩色图像，分别有$R、G、B$三个通道，则输入通道数为3。\n",
    "\n",
    "此外，由于具有单个核的卷积每次只能提取一种类型的特征，即输出一张大小为$U\\times V$的**特征图**（Feature Map）。而在实际应用中，我们也希望每一个卷积层能够提取多种不同类型的特征，所以一个卷积层通常会组合多个不同的卷积核来提取特征，经过卷积运算后会输出多张特征图，不同的特征图对应不同类型的特征。输出特征图的个数通常将其称为**输出通道数**$P$。\n",
    "\n",
    "------\n",
    "**说明：**\n",
    "\n",
    "《神经网络与深度学习》将Feature Map翻译为“特征映射”，这里翻译为“特征图”。\n",
    "\n",
    "------\n",
    "\n",
    "假设一个卷积层的输入特征图$\\mathbf X\\in \\mathbb{R}^{D\\times M\\times N}$，其中$(M,N)$为特征图的尺寸，$D$代表通道数；卷积核为$\\mathbf W\\in \\mathbb{R}^{P\\times D\\times U\\times V}$，其中$(U,V)$为卷积核的尺寸，$D$代表输入通道数，$P$代表输出通道数。\n",
    "\n",
    "------\n",
    "**说明：**\n",
    "\n",
    "在实践中，根据目前深度学习框架中张量的组织和运算性质，这里特征图的大小为$D\\times M\\times N$，和《神经网络与深度学习》中$M\\times N \\times D$的定义并不一致。\n",
    "相应地，卷积核$W$的大小为$\\mathbb{R}^{P\\times D\\times U\\times V}$。\n",
    "\n",
    "------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**一张输出特征图的计算**\n",
    "\n",
    "对于$D$个输入通道，分别对每个通道的特征图$\\mathbf X^d$设计一个二维卷积核$\\mathbf W^{p,d}$，并与对应的输入特征图$\\mathbf X^d$进行卷积运算，再将得到的$D$个结果进行加和，得到一张输出特征图$\\mathbf Z^p$。计算方式如下：\n",
    "\n",
    "$$\n",
    "\\mathbf Z^p = \\sum_{d=1}^D \\mathbf W^{p,d} \\otimes \\mathbf X^d + b^p，（5.18）\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\mathbf Y^p = f(\\mathbf Z^p)。（5.19）\n",
    "$$\n",
    "\n",
    "其中$p$表示输出特征图的索引编号，$\\mathbf W^{p,d} \\in \\mathbb{R}^{U\\times V}$为二维卷积核，$b^p$为标量偏置，$f(·)$为非线性激活函数，一般用ReLU函数。\n",
    "\n",
    "*******\n",
    "\n",
    "说明：\n",
    "\n",
    "在代码实现时，通常将非线性激活函数放在卷积层算子外部。\n",
    "\n",
    "*******\n",
    "\n",
    "公式（5.13）对应的可视化如**图5.9**所示。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/d4d1434e62154505a56259d09c9b6e072f8331830c2b4ffc97b5d7bd0181b075\" width = \"600\"></center>\n",
    "<center><br>图5.9：多输入通道的卷积运算 </br></center>\n",
    "<br></br>\n",
    "\n",
    "**多张输出特征图的计算**\n",
    "\n",
    "对于大小为$D\\times M\\times N$的输入特征图，每一个输出特征图都需要一组大小为$\\mathbf W\\in \\mathbb{R}^{D\\times U\\times V}$的卷积核进行卷积运算。使用$P$组卷积核分布进行卷积运算，得到$P$个输出特征图$\\mathbf Y^1, \\mathbf Y^2,\\cdots,\\mathbf Y^P$。然后将$P$个输出特征图进行拼接，获得大小为$P\\times M' \\times N'$的多通道输出特征图。上面计算方式的可视化如下**图5.10**所示。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/7a75788ae8d54f22882590a0923f5dd28bca6e120b1a4431abe122980d06224f\" width = \"600\"></center>\n",
    "<center><br>图5.10：多输出通道的卷积运算 </br></center>\n",
    "<br></br>\n",
    "\n",
    "#### 5.2.1.2 多通道卷积层算子\n",
    "\n",
    "根据上面的公式，多通道卷积卷积层的代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "inputs shape: [1, 2, 3, 3]\n",
      "Conv2D outputs shape: [1, 3, 2, 2]\n",
      "Conv2D outputs: Tensor(shape=[1, 3, 2, 2], dtype=float32, place=CUDAPlace(0), stop_gradient=False,\n",
      "       [[[[20., 28.],\n",
      "          [44., 52.]],\n",
      "\n",
      "         [[20., 28.],\n",
      "          [44., 52.]],\n",
      "\n",
      "         [[20., 28.],\n",
      "          [44., 52.]]]])\n",
      "nn.Conv2D outputs: Tensor(shape=[1, 3, 2, 2], dtype=float32, place=CUDAPlace(0), stop_gradient=False,\n",
      "       [[[[20., 28.],\n",
      "          [44., 52.]],\n",
      "\n",
      "         [[20., 28.],\n",
      "          [44., 52.]],\n",
      "\n",
      "         [[20., 28.],\n",
      "          [44., 52.]]]])\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. \n",
      "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
      "  if data.dtype == np.object:\n"
     ]
    }
   ],
   "source": [
    "class Conv2D(nn.Layer):\n",
    "    def __init__(self, in_channels, out_channels, kernel_size, stride=1, padding=0,\n",
    "                    weight_attr=paddle.ParamAttr(initializer=nn.initializer.Constant(value=1.0)),\n",
    "                    bias_attr=paddle.ParamAttr(initializer=nn.initializer.Constant(value=0.0))):\n",
    "        super(Conv2D, self).__init__()\n",
    "        # 创建卷积核\n",
    "        self.weight = paddle.create_parameter(shape=[out_channels, in_channels, kernel_size,kernel_size],\n",
    "                                                dtype='float32',\n",
    "                                                attr=weight_attr)\n",
    "        # 创建偏置\n",
    "        self.bias = paddle.create_parameter(shape=[out_channels, 1],\n",
    "                                                dtype='float32',\n",
    "                                                attr=bias_attr)\n",
    "        self.stride = stride\n",
    "        self.padding = padding\n",
    "        # 输入通道数\n",
    "        self.in_channels = in_channels\n",
    "        # 输出通道数\n",
    "        self.out_channels = out_channels\n",
    "\n",
    "    # 基础卷积运算\n",
    "    def single_forward(self, X, weight):\n",
    "        # 零填充\n",
    "        new_X = paddle.zeros([X.shape[0], X.shape[1]+2*self.padding, X.shape[2]+2*self.padding])\n",
    "        new_X[:, self.padding:X.shape[1]+self.padding, self.padding:X.shape[2]+self.padding] = X\n",
    "        u, v = weight.shape\n",
    "        output_w = (new_X.shape[1] - u) // self.stride + 1\n",
    "        output_h = (new_X.shape[2] - v) // self.stride + 1\n",
    "        output = paddle.zeros([X.shape[0], output_w, output_h])\n",
    "        for i in range(0, output.shape[1]):\n",
    "            for j in range(0, output.shape[2]):\n",
    "                output[:, i, j] = paddle.sum(\n",
    "                    new_X[:, self.stride*i:self.stride*i+u, self.stride*j:self.stride*j+v]*weight, \n",
    "                    axis=[1,2])\n",
    "        return output\n",
    "\n",
    "    def forward(self, inputs):\n",
    "        \"\"\"\n",
    "        输入：\n",
    "            - inputs：输入矩阵，shape=[B, D, M, N]\n",
    "            - weights：P组二维卷积核，shape=[P, D, U, V]\n",
    "            - bias：P个偏置，shape=[P, 1]\n",
    "        \"\"\"\n",
    "        feature_maps = []\n",
    "        # 进行多次多输入通道卷积运算\n",
    "        p=0\n",
    "        for w, b in zip(self.weight, self.bias): # P个(w,b),每次计算一个特征图Zp\n",
    "            multi_outs = []\n",
    "            # 循环计算每个输入特征图对应的卷积结果\n",
    "            for i in range(self.in_channels):\n",
    "                single = self.single_forward(inputs[:,i,:,:], w[i])\n",
    "                multi_outs.append(single)\n",
    "                # print(\"Conv2D in_channels:\",self.in_channels,\"i:\",i,\"single:\",single.shape)\n",
    "            # 将所有卷积结果相加\n",
    "            feature_map = paddle.sum(paddle.stack(multi_outs), axis=0) + b #Zp\n",
    "            feature_maps.append(feature_map)\n",
    "            # print(\"Conv2D out_channels:\",self.out_channels, \"p:\",p,\"feature_map:\",feature_map.shape)\n",
    "            p+=1\n",
    "        # 将所有Zp进行堆叠\n",
    "        out = paddle.stack(feature_maps, 1) \n",
    "        return out\n",
    "\n",
    "inputs = paddle.to_tensor([[[[0.0, 1.0, 2.0], [3.0, 4.0, 5.0], [6.0, 7.0, 8.0]],\n",
    "               [[1.0, 2.0, 3.0], [4.0, 5.0, 6.0], [7.0, 8.0, 9.0]]]])\n",
    "conv2d = Conv2D(in_channels=2, out_channels=3, kernel_size=2)\n",
    "print(\"inputs shape:\",inputs.shape)\n",
    "outputs = conv2d(inputs)\n",
    "print(\"Conv2D outputs shape:\",outputs.shape)\n",
    "\n",
    "# 比较与paddle API运算结果\n",
    "conv2d_paddle = nn.Conv2D(in_channels=2, out_channels=3, kernel_size=2,\n",
    "                            weight_attr=paddle.ParamAttr(initializer=nn.initializer.Constant(value=1.0)),\n",
    "                            bias_attr=paddle.ParamAttr(initializer=nn.initializer.Constant(value=0.0)))\n",
    "outputs_paddle = conv2d_paddle(inputs)\n",
    "# 自定义算子运算结果\n",
    "print('Conv2D outputs:', outputs)\n",
    "# paddle API运算结果\n",
    "print('nn.Conv2D outputs:', outputs_paddle)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 5.2.1.3 卷积算子的参数量和计算量\n",
    "\n",
    "**参数量**\n",
    "\n",
    "对于大小为$D\\times M\\times N$的输入特征图，使用$P$组大小为$\\mathbf W\\in \\mathbb{R}^{D\\times U\\times V}$的卷积核进行卷积运算，参数量计算方式为：\n",
    "\n",
    "$$\n",
    "parameters = P \\times D \\times U \\times V + P.（5.20）\n",
    "$$\n",
    "\n",
    "其中，最后的$P$代表偏置个数。例如：输入特征图大小为$3\\times 32\\times 32$，使用$6$组大小为$3\\times 3\\times 3$的卷积核进行卷积运算，参数量为：\n",
    "$$\n",
    "parameters = 6 \\times 3 \\times 3 \\times 3 + 6= 168.\n",
    "$$\n",
    "\n",
    "**计算量**\n",
    "\n",
    "对于大小为$D\\times M\\times N$的输入特征图，使用$P$组大小为$\\mathbf W\\in \\mathbb{R}^{D\\times U\\times V}$的卷积核进行卷积运算，计算量计算方式为：\n",
    "\n",
    "$$FLOPs=M'\\times N'\\times P\\times D\\times U\\times V + M'\\times N'\\times P。（5.21）$$\n",
    "\n",
    "其中$M'\\times N'\\times P$代表加偏置的计算量，即输出特征图上每个点都要与$P$组卷积核$\\mathbf W\\in \\mathbb{R}^{D\\times U\\times V}$进行$U\\times V\\times D$次乘法运算后再加上偏置。比如对于输入特征图大小为$3\\times 32\\times 32$，使用$6$组大小为$3\\times 3\\times 3$的卷积核进行卷积运算，计算量为：\n",
    "\n",
    "$$FLOPs=M'\\times N'\\times P\\times D\\times U\\times V + M'\\times N'\\times P= 30\\times 30\\times 3\\times 3\\times 6\\times 3 + 30\\times 30\\times 6= 151200$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.2.2 汇聚层算子\n",
    "\n",
    "**汇聚层**的作用是进行特征选择，降低特征数量，从而减少参数数量。由于汇聚之后特征图会变得更小，如果后面连接的是全连接层，可以有效地减小神经元的个数，节省存储空间并提高计算效率。\n",
    "\n",
    "常用的汇聚方法有两种，分别是：平均汇聚和最大汇聚。\n",
    "\n",
    "* 平均汇聚：将输入特征图划分为$2\\times2$大小的区域，对每个区域内的神经元活性值取平均值作为这个区域的表示；\n",
    "* 最大汇聚：使用输入特征图的每个子区域内所有神经元的最大活性值作为这个区域的表示。\n",
    "\n",
    "**图5.11** 给出了两种汇聚层的示例。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/d55501bb93914955b1c262a00201f8bcc6abf632085e48e3b37aac5f064a4e1c\" width = \"600\"></center>\n",
    "<center><br>图5.11：汇聚层 </br></center>\n",
    "<br></br>\n",
    "\n",
    "汇聚层输出的计算尺寸与卷积层一致，对于一个输入矩阵$\\mathbf X\\in\\Bbb{R}^{M\\times N}$和一个运算区域大小为$U\\times V$的汇聚层，步长为$S$，对输入矩阵进行零填充，那么最终输出矩阵大小则为\n",
    "\n",
    "$$M' = \\frac{M + 2P - U}{S} + 1,（5.20）$$\n",
    "$$N' = \\frac{N + 2P - V}{S} + 1.（5.21）$$\n",
    "\n",
    "由于过大的采样区域会急剧减少神经元的数量，也会造成过多的信息丢失。目前，在卷积神经网络中比较典型的汇聚层是将每个输入特征图划分为$2\\times2$大小的不重叠区域，然后使用最大汇聚的方式进行下采样。\n",
    "\n",
    "由于汇聚是使用某一位置的相邻输出的总体统计特征代替网络在该位置的输出，所以其好处是当输入数据做出少量平移时，经过汇聚运算后的大多数输出还能保持不变。比如：当识别一张图像是否是人脸时，我们需要知道人脸左边有一只眼睛，右边也有一只眼睛，而不需要知道眼睛的精确位置，这时候通过汇聚某一片区域的像素点来得到总体统计特征会显得很有用。这也就体现了汇聚层的**平移不变**特性。\n",
    "\n",
    "**汇聚层的参数量和计算量**\n",
    "\n",
    "由于汇聚层中没有参数，所以参数量为$0$；最大汇聚中，没有乘加运算，所以计算量为$0$，而平均汇聚中，输出特征图上每个点都对应了一次求平均运算。\n",
    "\n",
    "使用飞桨实现一个简单的汇聚层，代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input: [1, 1, 4, 4], \n",
      "output: [1, 1, 2, 2]\n",
      "Maxpool2D outputs: Tensor(shape=[1, 1, 2, 2], dtype=float32, place=CUDAPlace(0), stop_gradient=True,\n",
      "       [[[[6. , 8. ],\n",
      "          [14., 16.]]]])\n",
      "nn.Maxpool2D outputs: Tensor(shape=[1, 1, 2, 2], dtype=float32, place=CUDAPlace(0), stop_gradient=True,\n",
      "       [[[[6. , 8. ],\n",
      "          [14., 16.]]]])\n",
      "Avgpool2D outputs: Tensor(shape=[1, 1, 2, 2], dtype=float32, place=CUDAPlace(0), stop_gradient=True,\n",
      "       [[[[3.50000000 , 5.50000000 ],\n",
      "          [11.50000000, 13.50000000]]]])\n",
      "nn.Avgpool2D outputs: Tensor(shape=[1, 1, 2, 2], dtype=float32, place=CUDAPlace(0), stop_gradient=True,\n",
      "       [[[[3.50000000 , 5.50000000 ],\n",
      "          [11.50000000, 13.50000000]]]])\n"
     ]
    }
   ],
   "source": [
    "class Pool2D(nn.Layer):\n",
    "    def __init__(self, size=(2,2), mode='max', stride=1):\n",
    "        super(Pool2D, self).__init__()\n",
    "        # 汇聚方式\n",
    "        self.mode = mode\n",
    "        self.h, self.w = size\n",
    "        self.stride = stride\n",
    "\n",
    "    def forward(self, x):\n",
    "        output_w = (x.shape[2] - self.w) // self.stride + 1\n",
    "        output_h = (x.shape[3] - self.h) // self.stride + 1\n",
    "        output = paddle.zeros([x.shape[0], x.shape[1], output_w, output_h])\n",
    "        # 汇聚\n",
    "        for i in range(output.shape[2]):\n",
    "            for j in range(output.shape[3]):\n",
    "                # 最大汇聚\n",
    "                if self.mode == 'max':\n",
    "                    output[:, :, i, j] = paddle.max(\n",
    "                        x[:, :, self.stride*i:self.stride*i+self.w, self.stride*j:self.stride*j+self.h], \n",
    "                        axis=[2,3])\n",
    "                # 平均汇聚\n",
    "                elif self.mode == 'avg':\n",
    "                    output[:, :, i, j] = paddle.mean(\n",
    "                        x[:, :, self.stride*i:self.stride*i+self.w, self.stride*j:self.stride*j+self.h], \n",
    "                        axis=[2,3])\n",
    "        \n",
    "        return output\n",
    "\n",
    "inputs = paddle.to_tensor([[[[1.,2.,3.,4.],[5.,6.,7.,8.],[9.,10.,11.,12.],[13.,14.,15.,16.]]]])\n",
    "pool2d = Pool2D(stride=2)\n",
    "outputs = pool2d(inputs)\n",
    "print(\"input: {}, \\noutput: {}\".format(inputs.shape, outputs.shape))\n",
    "\n",
    "# 比较Maxpool2D与paddle API运算结果\n",
    "maxpool2d_paddle = nn.MaxPool2D(kernel_size=(2,2), stride=2)\n",
    "outputs_paddle = maxpool2d_paddle(inputs)\n",
    "# 自定义算子运算结果\n",
    "print('Maxpool2D outputs:', outputs)\n",
    "# paddle API运算结果\n",
    "print('nn.Maxpool2D outputs:', outputs_paddle)\n",
    "\n",
    "# 比较Avgpool2D与paddle API运算结果\n",
    "avgpool2d_paddle = nn.AvgPool2D(kernel_size=(2,2), stride=2)\n",
    "outputs_paddle = avgpool2d_paddle(inputs)\n",
    "pool2d = Pool2D(mode='avg', stride=2)\n",
    "outputs = pool2d(inputs)\n",
    "# 自定义算子运算结果\n",
    "print('Avgpool2D outputs:', outputs)\n",
    "# paddle API运算结果\n",
    "print('nn.Avgpool2D outputs:', outputs_paddle)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 5.3 基于LeNet实现手写体数字识别实验\n",
    "\n",
    "在本节中，我们实现经典卷积网络LeNet-5，并进行手写体数字识别任务。\n",
    "\n",
    "### 5.3.1 数据\n",
    "\n",
    "手写体数字识别是计算机视觉中最常用的图像分类任务，让计算机识别出给定图片中的手写体数字（0-9共10个数字）。由于手写体风格差异很大，因此手写体数字识别是具有一定难度的任务。\n",
    "\n",
    "我们采用常用的手写数字识别数据集：**MNIST数据集**。MNIST数据集是计算机视觉领域的经典入门数据集，包含了60,000个训练样本和10,000个测试样本。这些数字已经过尺寸标准化并位于图像中心，图像是固定大小($28\\times28$像素)。**图5.12**给出了部分样本的示例。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/53774cf008e941d3b93ce2b8731a1089fc64539bebd347e3a255c2d8fb5ec1a9\" width=\"600\" hegiht=\"\" ></center>\n",
    "<center><br>图5.12：MNIST数据集示例</br></center>\n",
    "<br></br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "为了节省训练时间，本节选取MNIST数据集的一个子集进行后续实验，数据集的划分为：\n",
    "\n",
    "* 训练集：1,000条样本\n",
    "* 验证集：200条样本\n",
    "* 测试集：200条样本\n",
    "\n",
    "MNIST数据集分为train_set、dev_set和test_set三个数据集，每个数据集含两个列表分别存放了图片数据以及标签数据。比如train_set包含：\n",
    "\n",
    "* **图片数据**：[1 000, 784]的二维列表，包含1 000张图片。每张图片用一个长度为784的向量表示，内容是 $28\\times 28$ 尺寸的像素灰度值（黑白图片）。\n",
    "* **标签数据**：[1 000, 1]的列表，表示这些图片对应的分类标签，即0~9之间的数字。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "观察数据集分布情况，代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of train/dev/test set:1000/200/200\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "import gzip\n",
    "\n",
    "# 打印并观察数据集分布情况\n",
    "train_set, dev_set, test_set = json.load(gzip.open('./mnist.json.gz'))\n",
    "train_images, train_labels = train_set[0][:1000], train_set[1][:1000]\n",
    "dev_images, dev_labels = dev_set[0][:200], dev_set[1][:200]\n",
    "test_images, test_labels = test_set[0][:200], test_set[1][:200]\n",
    "train_set, dev_set, test_set = [train_images, train_labels], [dev_images, dev_labels], [test_images, test_labels]\n",
    "print('Length of train/dev/test set:{}/{}/{}'.format(len(train_set[0]), len(dev_set[0]), len(test_set[0])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "可视化观察其中的一张样本以及对应的标签，代码如下所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The number in the picture is 5\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATUAAAEyCAYAAACbGke8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAADRFJREFUeJzt3VGIXYWdx/Hfb9W+qA+RDCGk2Z2uyIIsbCxDKFSKS7dFfYm+SPNQslCIDwoKfVjxpb4syFJtXxYhYmgWrKWgrnmQ3YoIbmERJxI0GrqKRJowJld80D4V9bcPc4RpmDv3zD3nzrn3P98PhLlz7p3c/8mRr+fec84dJxEAVPFXQw8AAH0iagBKIWoASiFqAEohagBKIWoASiFqAEohagBKIWoASrl2J59s7969WV5e3smnBFDEmTNnPkmyNOlxOxq15eVlra6u7uRTAijC9kdtHtfp5aftO23/wfYHth/p8ncBQB+mjprtayT9u6S7JN0q6ajtW/saDACm0WVP7bCkD5J8mOTPkn4j6Ug/YwHAdLpE7YCkP274/mKz7C/YPm571fbqaDTq8HQAMNnMT+lIciLJSpKVpaWJBy4AoJMuUbsk6eCG77/ZLAOAwXSJ2puSbrH9LdvfkPQjSaf7GQsApjP1eWpJvrD9oKT/lnSNpJNJ3u1tMgCYQqeTb5O8LOnlnmYBgM649hNAKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQCnXDj0AFpvtoUdAj5IMPUJnnaJm+4KkzyV9KemLJCt9DAUA0+pjT+0fk3zSw98DAJ3xnhqAUrpGLZJ+Z/uM7eObPcD2cdurtldHo1HHpwOArXWN2u1Jvi3pLkkP2P7e1Q9IciLJSpKVpaWljk8HAFvrFLUkl5qvVyS9KOlwH0MBwLSmjprt623f+PVtST+UdK6vwQBgGl2Ofu6T9GJzntK1kn6d5L96mQoApjR11JJ8KOkfepwFY3CCK9Aep3QAKIWoASiFqAEohagBKIWoASiFqAEohagBKIWoASiFqAEohY/zHhBXCmCnVPiY7rbYUwNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKVxQALeymM/IXHXtqAEohagBKIWoASiFqAEohagBKIWoASiFqAEohagBK4eTbAbU9oZOP/f5Lbf7d+DfbvdhTA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApXFCyA3XIGfZ8fmc3Hb+9e7KkBKGVi1GyftH3F9rkNy26y/Yrt95uve2Y7JgC002ZP7VeS7rxq2SOSXk1yi6RXm+8BYHATo5bkdUmfXrX4iKRTze1Tku7peS4AmMq076ntS7LW3P5Y0r5xD7R93Paq7dXRaDTl0wFAO50PFGT9MNPYQ01JTiRZSbKytLTU9ekAYEvTRu2y7f2S1Hy90t9IADC9aaN2WtKx5vYxSS/1Mw4AdNPmlI7nJP2vpL+zfdH2TyQ9LukHtt+X9E/N9wAwuIlXFCQ5Ouau7/c8Czro+wz6Ia5QaPucXC2ArXBFAYBSiBqAUogagFKIGoBSiBqAUogagFKIGoBSiBqAUogagFKIGoBSiBqAUogagFKIGoBSiBqAUogagFKIGoBSiBqAUogagFImfpw3dqe2H5k9zx/73Sc+QnxxsKcGoBSiBqAUogagFKIGoBSiBqAUogagFKIGoBSiBqAUogagFK4oQCdtzrQf4gqAvrVdB648GB57agBKIWoASiFqAEohagBKIWoASiFqAEohagBKIWoASiFqAErhigLM3Dz/voO+tVkHrjqYrYl7arZP2r5i+9yGZY/ZvmT7bPPn7tmOCQDttHn5+StJd26y/BdJDjV/Xu53LACYzsSoJXld0qc7MAsAdNblQMGDtt9uXp7uGfcg28dtr9peHY1GHZ4OACabNmpPSbpZ0iFJa5KeGPfAJCeSrCRZWVpamvLpAKCdqaKW5HKSL5N8JelpSYf7HQsApjNV1Gzv3/DtvZLOjXssAOykieep2X5O0h2S9tq+KOlnku6wfUhSJF2QdP8MZwSA1iZGLcnRTRY/M4NZsMvtppN0MTtcJgWgFKIGoBSiBqAUogagFKIGoBSiBqAUogagFKIGoBSiBqAUPs4bC6fPj8Me4uqEts/Jx35Phz01AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApXBFAeYGv3sAfWBPDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClcUYBOuApg+/jdA7PFnhqAUogagFKIGoBSiBqAUogagFKIGoBSiBqAUogagFI4+XaX4WTZ2eLE2uGxpwaglIlRs33Q9mu237P9ru2HmuU32X7F9vvN1z2zHxcAttZmT+0LST9Ncquk70h6wPatkh6R9GqSWyS92nwPAIOaGLUka0neam5/Lum8pAOSjkg61TzslKR7ZjUkALS1rffUbC9Luk3SG5L2JVlr7vpY0r4xP3Pc9qrt1dFo1GFUAJisddRs3yDpeUkPJ/ls431ZP+Sz6WGfJCeSrCRZWVpa6jQsAEzSKmq2r9N60J5N8kKz+LLt/c39+yVdmc2IANBem6OflvSMpPNJntxw12lJx5rbxyS91P94ALA9bU6+/a6kH0t6x/bZZtmjkh6X9FvbP5H0kaT7ZjMiALQ3MWpJfi9p3Gno3+93HGyGqwCGx5UCi4MrCgCUQtQAlELUAJRC1ACUQtQAlELUAJRC1ACUQtQAlELUAJTC7yiYAa4AGB5XAOxe7KkBKIWoASiFqAEohagBKIWoASiFqAEohagBKIWoASiFk28bnDA7PE6YRR/YUwNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKVxSgE64CwLxhTw1AKUQNQClEDUApRA1AKUQNQClEDUApRA1AKUQNQClEDUApXFHQ4Mx4oIaJe2q2D9p+zfZ7tt+1/VCz/DHbl2yfbf7cPftxAWBrbfbUvpD00yRv2b5R0hnbrzT3/SLJz2c3HgBsz8SoJVmTtNbc/tz2eUkHZj0YAExjWwcKbC9Luk3SG82iB22/bfuk7T09zwYA29Y6arZvkPS8pIeTfCbpKUk3Szqk9T25J8b83HHbq7ZXR6NRDyMDwHitomb7Oq0H7dkkL0hSkstJvkzylaSnJR3e7GeTnEiykmRlaWmpr7kBYFNtjn5a0jOSzid5csPy/Rsedq+kc/2PBwDb0+bo53cl/VjSO7bPNsselXTU9iFJkXRB0v0zmRAAtqHN0c/fS/Imd73c/zgA0A2XSQEohagBKIWoASiFqAEohagBKIWoASiFqAEohagBKIWoASiFqAEohagBKIWoASiFqAEohagBKIWoASiFqAEohagBKIWoASiFqAEoxUl27snskaSPrlq8V9InOzZE/xZ9fmnx12HR55cWfx12Yv6/STLx92zuaNQ2HcBeTbIy6BAdLPr80uKvw6LPLy3+OszT/Lz8BFAKUQNQyjxE7cTQA3S06PNLi78Oiz6/tPjrMDfzD/6eGgD0aR721ACgN0QNQCmDRc32nbb/YPsD248MNUcXti/Yfsf2WdurQ8/Thu2Ttq/YPrdh2U22X7H9fvN1z5AzbmXM/I/ZvtRsh7O27x5yxq3YPmj7Ndvv2X7X9kPN8kXaBuPWYS62wyDvqdm+RtL/SfqBpIuS3pR0NMl7Oz5MB7YvSFpJsjAnTdr+nqQ/SfqPJH/fLPs3SZ8mebz5H8yeJP8y5JzjjJn/MUl/SvLzIWdrw/Z+SfuTvGX7RklnJN0j6Z+1ONtg3DrcpznYDkPtqR2W9EGSD5P8WdJvJB0ZaJZdJcnrkj69avERSaea26e0/h/oXBoz/8JIspbkreb255LOSzqgxdoG49ZhLgwVtQOS/rjh+4uao3+UbYik39k+Y/v40MN0sC/JWnP7Y0n7hhxmSg/afrt5eTq3L902sr0s6TZJb2hBt8FV6yDNwXbgQEE3tyf5tqS7JD3QvDRaaFl/P2LRzvN5StLNkg5JWpP0xLDjTGb7BknPS3o4yWcb71uUbbDJOszFdhgqapckHdzw/TebZQslyaXm6xVJL2r9ZfUiuty8T/L1+yVXBp5nW5JcTvJlkq8kPa053w62r9N6DJ5N8kKzeKG2wWbrMC/bYaiovSnpFtvfsv0NST+SdHqgWaZi+/rmTVLZvl7SDyWd2/qn5tZpScea28ckvTTgLNv2dQwa92qOt4NtS3pG0vkkT264a2G2wbh1mJftMNgVBc3h3l9KukbSyST/OsggU7L9t1rfO5OkayX9ehHWwfZzku7Q+kfFXJb0M0n/Kem3kv5a6x8NdV+SuXwzfsz8d2j9JU8kXZB0/4b3p+aK7dsl/Y+kdyR91Sx+VOvvSS3KNhi3Dkc1B9uBy6QAlMKBAgClEDUApRA1AKUQNQClEDUApRA1AKUQNQCl/D9rTFOGLihehQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "image, label = train_set[0][0], train_set[1][0]\n",
    "image, label = np.array(image).astype('float32'), int(label)\n",
    "# 原始图像数据为长度784的行向量，需要调整为[28,28]大小的图像\n",
    "image = np.reshape(image, [28,28])\n",
    "image = Image.fromarray(image.astype('uint8'), mode='L')\n",
    "print(\"The number in the picture is {}\".format(label))\n",
    "plt.figure(figsize=(5, 5))\n",
    "plt.imshow(image)\n",
    "plt.savefig('conv-number5.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 5.3.1.1 数据预处理\n",
    "\n",
    "图像分类网络对输入图片的格式、大小有一定的要求，数据输入模型前，需要对数据进行预处理操作，使图片满足网络训练以及预测的需要。本实验主要应用了如下方法：\n",
    "\n",
    "*  调整图片大小：LeNet网络对输入图片大小的要求为 $32\\times 32$ ，而MNIST数据集中的原始图片大小却是 $28\\times 28$ ，这里为了符合网络的结构设计，将其调整为$32 \\times 32$；\n",
    "*  规范化： 通过规范化手段，把输入图像的分布改变成均值为0，标准差为1的标准正态分布，使得最优解的寻优过程明显会变得平缓，训练过程更容易收敛。\n",
    "\n",
    "*******\n",
    "\n",
    "在飞桨中，提供了部分视觉领域的高层API，可以直接调用API实现简单的图像处理操作。通过调用`paddle.vision.transforms.Resize`调整大小；调用`paddle.vision.transforms.Normalize`进行标准化处理；使用`paddle.vision.transforms.Compose`将两个预处理操作进行拼接。\n",
    "\n",
    "*******\n",
    "\n",
    "代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from paddle.vision.transforms import Compose, Resize, Normalize\n",
    "\n",
    "# 数据预处理\n",
    "transforms = Compose([Resize(32), Normalize(mean=[127.5], std=[127.5], data_format='CHW')])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "将原始的数据集封装为Dataset类，以便DataLoader调用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import random\n",
    "import paddle.io as io\n",
    "\n",
    "class MNIST_dataset(io.Dataset):\n",
    "    def __init__(self, dataset, transforms, mode='train'):\n",
    "        self.mode = mode\n",
    "        self.transforms =transforms\n",
    "        self.dataset = dataset\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        # 获取图像和标签\n",
    "        image, label = self.dataset[0][idx], self.dataset[1][idx]\n",
    "        image, label = np.array(image).astype('float32'), int(label)\n",
    "        image = np.reshape(image, [28,28])\n",
    "        image = Image.fromarray(image.astype('uint8'), mode='L')\n",
    "        image = self.transforms(image)\n",
    "\n",
    "        return image, label\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.dataset[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 固定随机种子\n",
    "random.seed(0)\n",
    "# 加载 mnist 数据集\n",
    "train_dataset = MNIST_dataset(dataset=train_set, transforms=transforms, mode='train')\n",
    "test_dataset = MNIST_dataset(dataset=test_set, transforms=transforms, mode='test')\n",
    "dev_dataset = MNIST_dataset(dataset=dev_set, transforms=transforms, mode='dev')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.3.2 模型构建\n",
    "\n",
    "LeNet-5虽然提出的时间比较早，但它是一个非常成功的神经网络模型。基于LeNet-5的手写数字识别系统在20世纪90年代被美国很多银行使用，用来识别支票上面的手写数字。LeNet-5的网络结构如**图5.13**所示。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/448b1f120a2e4890bae4589d213c0ab36f71b8d9d6204998a6cceddaabbe9ab5\" width=\"600\" hegiht=\"\" ></center>\n",
    "<center><br>图5.13：LeNet-5网络结构</br></center>\n",
    "<br></br>\n",
    "\n",
    "我们使用上面定义的卷积层算子和汇聚层算子构建一个LeNet-5模型。\n",
    "\n",
    "************\n",
    "\n",
    "  这里的LeNet-5和原始版本有4点不同：\n",
    "1. C3层没有使用连接表来减少卷积数量。\n",
    "1. 汇聚层使用了简单的平均汇聚，没有引入权重和偏置参数以及非线性激活函数。\n",
    "1. 卷积层的激活函数使用ReLU函数。\n",
    "1. 最后的输出层为一个全连接线性层。\n",
    "\n",
    "************\n",
    "\n",
    "网络共有7层，包含3个卷积层、2个汇聚层以及2个全连接层的简单卷积神经网络接，受输入图像大小为$32\\times 32=1\\, 024$，输出对应10个类别的得分。\n",
    "具体实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import paddle.nn.functional as F\n",
    "\n",
    "class Model_LeNet(nn.Layer):\n",
    "    def __init__(self, in_channels, num_classes=10):\n",
    "        super(Model_LeNet, self).__init__()\n",
    "        # 卷积层：输出通道数为6，卷积核大小为5×5\n",
    "        self.conv1 = Conv2D(in_channels=in_channels, out_channels=6, kernel_size=5, weight_attr=paddle.ParamAttr())\n",
    "        # 汇聚层：汇聚窗口为2×2，步长为2\n",
    "        self.pool2 = Pool2D(size=(2,2), mode='max', stride=2)\n",
    "        # 卷积层：输入通道数为6，输出通道数为16，卷积核大小为5×5，步长为1\n",
    "        self.conv3 = Conv2D(in_channels=6, out_channels=16, kernel_size=5, stride=1, weight_attr=paddle.ParamAttr())\n",
    "        # 汇聚层：汇聚窗口为2×2，步长为2\n",
    "        self.pool4 = Pool2D(size=(2,2), mode='avg', stride=2)\n",
    "        # 卷积层：输入通道数为16，输出通道数为120，卷积核大小为5×5\n",
    "        self.conv5 = Conv2D(in_channels=16, out_channels=120, kernel_size=5, stride=1, weight_attr=paddle.ParamAttr())\n",
    "        # 全连接层：输入神经元为120，输出神经元为84\n",
    "        self.linear6 = nn.Linear(120, 84)\n",
    "        # 全连接层：输入神经元为84，输出神经元为类别数\n",
    "        self.linear7 = nn.Linear(84, num_classes)\n",
    "\n",
    "    def forward(self, x):\n",
    "        # C1：卷积层+激活函数\n",
    "        output = F.relu(self.conv1(x))\n",
    "        # S2：汇聚层\n",
    "        output = self.pool2(output)\n",
    "        # C3：卷积层+激活函数\n",
    "        output = F.relu(self.conv3(output))\n",
    "        # S4：汇聚层\n",
    "        output = self.pool4(output)\n",
    "        # C5：卷积层+激活函数\n",
    "        output = F.relu(self.conv5(output))\n",
    "        # 输入层将数据拉平[B,C,H,W] -> [B,CxHxW]\n",
    "        output = paddle.squeeze(output, axis=[2,3])\n",
    "        # F6：全连接层\n",
    "        output = F.relu(self.linear6(output))\n",
    "        # F7：全连接层\n",
    "        output = self.linear7(output)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "下面测试一下上面的LeNet-5模型，构造一个形状为 [1,1,32,32]的输入数据送入网络，观察每一层特征图的形状变化。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Conv2D(), Pool2D(), Conv2D(), Pool2D(), Conv2D(), Linear(in_features=120, out_features=84, dtype=float32), Linear(in_features=84, out_features=10, dtype=float32)]\n",
      "conv2d_6 [1, 6, 28, 28] [6, 1, 5, 5] [6, 1]\n",
      "pool2d_2 [1, 6, 14, 14]\n",
      "conv2d_7 [1, 16, 10, 10] [16, 6, 5, 5] [16, 1]\n",
      "pool2d_3 [1, 16, 5, 5]\n",
      "conv2d_8 [1, 120, 1, 1] [120, 16, 5, 5] [120, 1]\n",
      "linear_0 [1, 84] [120, 84] [84]\n",
      "linear_1 [1, 10] [84, 10] [10]\n"
     ]
    }
   ],
   "source": [
    "# 这里用np.random创建一个随机数组作为输入数据\n",
    "inputs = np.random.randn(*[1,1,32,32])\n",
    "inputs = inputs.astype('float32')\n",
    "\n",
    "# 创建Model_LeNet类的实例，指定模型名称和分类的类别数目\n",
    "model = Model_LeNet(in_channels=1, num_classes=10)\n",
    "# 通过调用LeNet从基类继承的sublayers()函数，查看LeNet中所包含的子层\n",
    "print(model.sublayers())\n",
    "x = paddle.to_tensor(inputs)\n",
    "for item in model.sublayers():\n",
    "    # item是LeNet类中的一个子层\n",
    "    # 查看经过子层之后的输出数据形状\n",
    "    try:\n",
    "        x = item(x)\n",
    "    except:\n",
    "        # 如果是最后一个卷积层输出，需要展平后才可以送入全连接层\n",
    "        x = paddle.reshape(x, [x.shape[0], -1])\n",
    "        x = item(x)\n",
    "    if len(item.parameters())==2:\n",
    "        # 查看卷积和全连接层的数据和参数的形状，\n",
    "        # 其中item.parameters()[0]是权重参数w，item.parameters()[1]是偏置参数b\n",
    "        print(item.full_name(), x.shape, item.parameters()[0].shape, \n",
    "                item.parameters()[1].shape)\n",
    "    else:\n",
    "        # 汇聚层没有参数\n",
    "        print(item.full_name(), x.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，\n",
    "* 对于大小为$32 \\times32$的单通道图像，先用6个大小为$5 \\times5$的卷积核对其进行卷积运算，输出为6个$28 \\times28$大小的特征图；\n",
    "* 6个$28 \\times28$大小的特征图经过大小为$2 \\times2$，步长为2的汇聚层后，输出特征图的大小变为$14 \\times14$；\n",
    "* 6个$14 \\times14$大小的特征图再经过16个大小为$5 \\times5$的卷积核对其进行卷积运算，得到16个$10 \\times10$大小的输出特征图；\n",
    "* 16个$10 \\times10$大小的特征图经过大小为$2 \\times2$，步长为2的汇聚层后，输出特征图的大小变为$5 \\times5$；\n",
    "* 16个$5 \\times5$大小的特征图再经过120个大小为$5 \\times5$的卷积核对其进行卷积运算，得到120个$1 \\times1$大小的输出特征图；\n",
    "* 此时，将特征图展平成1维，则有120个像素点，经过输入神经元个数为120，输出神经元个数为84的全连接层后，输出的长度变为84。\n",
    "* 再经过一个全连接层的计算，最终得到了长度为类别数的输出结果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "考虑到自定义的`Conv2D`和`Pool2D`算子中包含多个`for`循环，所以运算速度比较慢。飞桨框架中，针对卷积层算子和汇聚层算子进行了速度上的优化，这里基于`paddle.nn.Conv2D`、`paddle.nn.MaxPool2D`和`paddle.nn.AvgPool2D`构建LeNet-5模型，对比与上边实现的模型的运算速度。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class Paddle_LeNet(nn.Layer):\n",
    "    def __init__(self, in_channels, num_classes=10):\n",
    "        super(Paddle_LeNet, self).__init__()\n",
    "        # 卷积层：输出通道数为6，卷积核大小为5*5\n",
    "        self.conv1 = nn.Conv2D(in_channels=in_channels, out_channels=6, kernel_size=5)\n",
    "        # 汇聚层：汇聚窗口为2*2，步长为2\n",
    "        self.pool2 = nn.MaxPool2D(kernel_size=2, stride=2)\n",
    "        # 卷积层：输入通道数为6，输出通道数为16，卷积核大小为5*5\n",
    "        self.conv3 = nn.Conv2D(in_channels=6, out_channels=16, kernel_size=5)\n",
    "        # 汇聚层：汇聚窗口为2*2，步长为2\n",
    "        self.pool4 = nn.AvgPool2D(kernel_size=2, stride=2)\n",
    "        # 卷积层：输入通道数为16，输出通道数为120，卷积核大小为5*5\n",
    "        self.conv5 = nn.Conv2D(in_channels=16, out_channels=120, kernel_size=5)\n",
    "        # 全连接层：输入神经元为120，输出神经元为84\n",
    "        self.linear6 = nn.Linear(in_features=120, out_features=84)\n",
    "        # 全连接层：输入神经元为84，输出神经元为类别数\n",
    "        self.linear7 = nn.Linear(in_features=84, out_features=num_classes)\n",
    "\n",
    "    def forward(self, x):\n",
    "        # C1：卷积层+激活函数\n",
    "        output = F.relu(self.conv1(x))\n",
    "        # S2：汇聚层\n",
    "        output = self.pool2(output)\n",
    "        # C3：卷积层+激活函数\n",
    "        output = F.relu(self.conv3(output))\n",
    "        # S4：汇聚层\n",
    "        output = self.pool4(output)\n",
    "        # C5：卷积层+激活函数\n",
    "        output = F.relu(self.conv5(output))\n",
    "        # 输入层将数据拉平[B,C,H,W] -> [B,CxHxW]\n",
    "        output = paddle.squeeze(output, axis=[2,3])\n",
    "        # F6：全连接层\n",
    "        output = F.relu(self.linear6(output))\n",
    "        # F7：全连接层\n",
    "        output = self.linear7(output)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "测试两个网络的运算速度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_150/137857584.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     15\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m60\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     16\u001b[0m     \u001b[0mstrat_time\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 17\u001b[0;31m     \u001b[0mout\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     18\u001b[0m     \u001b[0mend_time\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     19\u001b[0m     \u001b[0;31m# 预热10次运算，不计入最终速度统计\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *inputs, **kwargs)\u001b[0m\n\u001b[1;32m    915\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    916\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 917\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_dygraph_call_func\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    918\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    919\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py\u001b[0m in \u001b[0;36m_dygraph_call_func\u001b[0;34m(self, *inputs, **kwargs)\u001b[0m\n\u001b[1;32m    905\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_built\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    906\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 907\u001b[0;31m         \u001b[0moutputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    908\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    909\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mforward_post_hook\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_forward_post_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/tmp/ipykernel_150/2647073283.py\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, x)\u001b[0m\n\u001b[1;32m     25\u001b[0m         \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpool2\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutput\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     26\u001b[0m         \u001b[0;31m# C3：卷积层+激活函数\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 27\u001b[0;31m         \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mF\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrelu\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconv3\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutput\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     28\u001b[0m         \u001b[0;31m# S4：汇聚层\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     29\u001b[0m         \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpool4\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutput\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *inputs, **kwargs)\u001b[0m\n\u001b[1;32m    915\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    916\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 917\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_dygraph_call_func\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    918\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    919\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py\u001b[0m in \u001b[0;36m_dygraph_call_func\u001b[0;34m(self, *inputs, **kwargs)\u001b[0m\n\u001b[1;32m    905\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_built\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    906\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 907\u001b[0;31m         \u001b[0moutputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    908\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    909\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mforward_post_hook\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_forward_post_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/tmp/ipykernel_150/870038972.py\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, inputs)\u001b[0m\n\u001b[1;32m     49\u001b[0m             \u001b[0;31m# 循环计算每个输入特征图对应的卷积结果\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     50\u001b[0m             \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0min_channels\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 51\u001b[0;31m                 \u001b[0msingle\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msingle_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mw\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     52\u001b[0m                 \u001b[0mmulti_outs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msingle\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     53\u001b[0m                 \u001b[0;31m# print(\"Conv2D in_channels:\",self.in_channels,\"i:\",i,\"single:\",single.shape)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/tmp/ipykernel_150/870038972.py\u001b[0m in \u001b[0;36msingle_forward\u001b[0;34m(self, X, weight)\u001b[0m\n\u001b[1;32m     31\u001b[0m             \u001b[0;32mfor\u001b[0m \u001b[0mj\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moutput\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     32\u001b[0m                 output[:, i, j] = paddle.sum(\n\u001b[0;32m---> 33\u001b[0;31m                     \u001b[0mnew_X\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstride\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstride\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mu\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstride\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mj\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstride\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mj\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mv\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mweight\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     34\u001b[0m                     axis=[1,2])\n\u001b[1;32m     35\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0moutput\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/varbase_patch_methods.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, item)\u001b[0m\n\u001b[1;32m    596\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    597\u001b[0m             \u001b[0;31m# 2. Call c++ func getitem_index_not_tensor to speedup.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 598\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getitem_index_not_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    599\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    600\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__setitem__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mitem\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "# 这里用np.random创建一个随机数组作为测试数据\n",
    "inputs = np.random.randn(*[1,1,32,32])\n",
    "inputs = inputs.astype('float32')\n",
    "x = paddle.to_tensor(inputs)\n",
    "\n",
    "# 创建Model_LeNet类的实例，指定模型名称和分类的类别数目\n",
    "model = Model_LeNet(in_channels=1, num_classes=10)\n",
    "# 创建Paddle_LeNet类的实例，指定模型名称和分类的类别数目\n",
    "paddle_model = Paddle_LeNet(in_channels=1, num_classes=10)\n",
    "\n",
    "# 计算Model_LeNet类的运算速度\n",
    "model_time = 0\n",
    "for i in range(60):\n",
    "    strat_time = time.time()\n",
    "    out = model(x)\n",
    "    end_time = time.time()\n",
    "    # 预热10次运算，不计入最终速度统计\n",
    "    if i < 10:\n",
    "        continue\n",
    "    model_time += (end_time - strat_time)\n",
    "avg_model_time = model_time / 50\n",
    "print('Model_LeNet speed:', avg_model_time, 's')\n",
    "\n",
    "# 计算Paddle_LeNet类的运算速度\n",
    "paddle_model_time = 0\n",
    "for i in range(60):\n",
    "    strat_time = time.time()\n",
    "    paddle_out = paddle_model(x)\n",
    "    end_time = time.time()\n",
    "    # 预热10次运算，不计入最终速度统计\n",
    "    if i < 10:\n",
    "        continue\n",
    "    paddle_model_time += (end_time - strat_time)\n",
    "avg_paddle_model_time = paddle_model_time / 50\n",
    "\n",
    "print('Paddle_LeNet speed:', avg_paddle_model_time, 's')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "这里还可以令两个网络加载同样的权重，测试一下两个网络的输出结果是否一致。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 这里用np.random创建一个随机数组作为测试数据\n",
    "inputs = np.random.randn(*[1,1,32,32])\n",
    "inputs = inputs.astype('float32')\n",
    "x = paddle.to_tensor(inputs)\n",
    "\n",
    "# 创建Model_LeNet类的实例，指定模型名称和分类的类别数目\n",
    "model = Model_LeNet(in_channels=1, num_classes=10)\n",
    "# 获取网络的权重\n",
    "params = model.state_dict()\n",
    "# 自定义Conv2D算子的bias参数形状为[out_channels, 1]\n",
    "# paddle API中Conv2D算子的bias参数形状为[out_channels]\n",
    "# 需要进行调整后才可以赋值\n",
    "for key in params:\n",
    "    if 'bias' in key:\n",
    "        params[key] = params[key].squeeze()\n",
    "# 创建Paddle_LeNet类的实例，指定模型名称和分类的类别数目\n",
    "paddle_model = Paddle_LeNet(in_channels=1, num_classes=10)\n",
    "# 将Model_LeNet的权重参数赋予给Paddle_LeNet模型，保持两者一致\n",
    "paddle_model.set_state_dict(params)\n",
    "\n",
    "# 打印结果保留小数点后6位\n",
    "paddle.set_printoptions(6)\n",
    "# 计算Model_LeNet的结果\n",
    "output = model(x)\n",
    "print('Model_LeNet output: ', output)\n",
    "# 计算Paddle_LeNet的结果\n",
    "paddle_output = paddle_model(x)\n",
    "print('Paddle_LeNet output: ', paddle_output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "可以看到，输出结果是一致的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "这里还可以统计一下LeNet-5模型的参数量和计算量。\n",
    "\n",
    "**参数量**\n",
    "\n",
    "按照公式(5.18)进行计算，可以得到：\n",
    "* 第一个卷积层的参数量为：$6 \\times 1 \\times 5 \\times 5 + 6 = 156$；\n",
    "* 第二个卷积层的参数量为：$16 \\times 6 \\times 5 \\times 5 + 16 = 2416$；\n",
    "* 第三个卷积层的参数量为：$120 \\times 16 \\times 5 \\times 5 + 120= 48120$；\n",
    "* 第一个全连接层的参数量为：$120 \\times 84 + 84= 10164$；\n",
    "* 第二个全连接层的参数量为：$84 \\times 10 + 10= 850$；\n",
    "\n",
    "所以，LeNet-5总的参数量为$61706$。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "在飞桨中，还可以使用`paddle.summary`API自动计算参数量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model = Paddle_LeNet(in_channels=1, num_classes=10)\n",
    "params_info = paddle.summary(model, (1, 1, 32, 32))\n",
    "print(params_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "可以看到，结果与公式推导一致。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**计算量**\n",
    "\n",
    "按照公式(5.19)进行计算，可以得到：\n",
    "* 第一个卷积层的计算量为：$28\\times 28\\times 5\\times 5\\times 6\\times 1 + 28\\times 28\\times 6=122304$；\n",
    "* 第二个卷积层的计算量为：$10\\times 10\\times 5\\times 5\\times 16\\times 6 + 10\\times 10\\times 16=241600$；\n",
    "* 第三个卷积层的计算量为：$1\\times 1\\times 5\\times 5\\times 120\\times 16 + 1\\times 1\\times 120=48120$；\n",
    "* 平均汇聚层的计算量为：$16\\times 5\\times 5=400$\n",
    "* 第一个全连接层的计算量为：$120 \\times 84 = 10080$；\n",
    "* 第二个全连接层的计算量为：$84 \\times 10 = 840$；\n",
    "\n",
    "所以，LeNet-5总的计算量为$423344$。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "在飞桨中，还可以使用`paddle.flops`API自动统计计算量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'paddle.nn.layer.conv.Conv2D'>'s flops has been counted\n",
      "Cannot find suitable count function for <class 'paddle.nn.layer.pooling.MaxPool2D'>. Treat it as zero FLOPs.\n",
      "<class 'paddle.nn.layer.pooling.AvgPool2D'>'s flops has been counted\n",
      "<class 'paddle.nn.layer.common.Linear'>'s flops has been counted\n",
      "+--------------+-----------------+-----------------+--------+--------+\n",
      "|  Layer Name  |   Input Shape   |   Output Shape  | Params | Flops  |\n",
      "+--------------+-----------------+-----------------+--------+--------+\n",
      "|  conv2d_21   |  [1, 1, 32, 32] |  [1, 6, 28, 28] |  156   | 122304 |\n",
      "| max_pool2d_3 |  [1, 6, 28, 28] |  [1, 6, 14, 14] |   0    |   0    |\n",
      "|  conv2d_22   |  [1, 6, 14, 14] | [1, 16, 10, 10] |  2416  | 241600 |\n",
      "| avg_pool2d_3 | [1, 16, 10, 10] |  [1, 16, 5, 5]  |   0    |  400   |\n",
      "|  conv2d_23   |  [1, 16, 5, 5]  |  [1, 120, 1, 1] | 48120  | 48120  |\n",
      "|  linear_10   |     [1, 120]    |     [1, 84]     | 10164  | 10080  |\n",
      "|  linear_11   |     [1, 84]     |     [1, 10]     |  850   |  840   |\n",
      "+--------------+-----------------+-----------------+--------+--------+\n",
      "Total Flops: 423344     Total Params: 61706\n",
      "423344\n"
     ]
    }
   ],
   "source": [
    "FLOPs = paddle.flops(model, (1, 1, 32, 32), print_detail=True)\n",
    "print(FLOPs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "可以看到，结果与公式推导一致。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.3.3 模型训练\n",
    "\n",
    "使用交叉熵损失函数，并用随机梯度下降法作为优化器来训练LeNet-5网络。\n",
    "用RunnerV3在训练集上训练5个epoch，并保存准确率最高的模型作为最佳模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Train] epoch: 0/5, step: 0/80, loss: 2.28455\n",
      "[Train] epoch: 0/5, step: 15/80, loss: 1.93600\n",
      "[Evaluate]  dev score: 0.38000, dev loss: 1.98255\n",
      "[Evaluate] best accuracy performence has been updated: 0.00000 --> 0.38000\n",
      "[Train] epoch: 1/5, step: 30/80, loss: 1.72429\n",
      "[Evaluate]  dev score: 0.63500, dev loss: 1.56296\n",
      "[Evaluate] best accuracy performence has been updated: 0.38000 --> 0.63500\n",
      "[Train] epoch: 2/5, step: 45/80, loss: 0.87373\n",
      "[Evaluate]  dev score: 0.72000, dev loss: 0.78202\n",
      "[Evaluate] best accuracy performence has been updated: 0.63500 --> 0.72000\n",
      "[Train] epoch: 3/5, step: 60/80, loss: 0.70869\n",
      "[Evaluate]  dev score: 0.75000, dev loss: 0.60713\n",
      "[Evaluate] best accuracy performence has been updated: 0.72000 --> 0.75000\n",
      "[Train] epoch: 4/5, step: 75/80, loss: 0.42060\n",
      "[Evaluate]  dev score: 0.87500, dev loss: 0.40498\n",
      "[Evaluate] best accuracy performence has been updated: 0.75000 --> 0.87500\n",
      "[Evaluate]  dev score: 0.75500, dev loss: 0.61064\n",
      "[Train] Training done!\n"
     ]
    }
   ],
   "source": [
    "import paddle.optimizer as opt\n",
    "from nndl import RunnerV3, metric\n",
    "\n",
    "paddle.seed(100)\n",
    "# 学习率大小\n",
    "lr = 0.1    \n",
    "# 批次大小\n",
    "batch_size = 64\n",
    "# 加载数据\n",
    "train_loader = io.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)\n",
    "dev_loader = io.DataLoader(dev_dataset, batch_size=batch_size)\n",
    "test_loader = io.DataLoader(test_dataset, batch_size=batch_size)\n",
    "# 定义LeNet网络\n",
    "# 自定义算子实现的LeNet-5\n",
    "model = Model_LeNet(in_channels=1, num_classes=10)\n",
    "# 飞桨API实现的LeNet-5\n",
    "# model = Paddle_LeNet(in_channels=1, num_classes=10)\n",
    "# 定义优化器\n",
    "optimizer = opt.SGD(learning_rate=lr, parameters=model.parameters())\n",
    "# 定义损失函数\n",
    "loss_fn = F.cross_entropy\n",
    "# 定义评价指标\n",
    "metric = metric.Accuracy(is_logist=True)\n",
    "# 实例化 RunnerV3 类，并传入训练配置。\n",
    "runner = RunnerV3(model, optimizer, loss_fn, metric)\n",
    "# 启动训练\n",
    "log_steps = 15\n",
    "eval_steps = 15\n",
    "runner.train(train_loader, dev_loader, num_epochs=5, log_steps=log_steps, \n",
    "                eval_steps=eval_steps, save_path=\"best_model.pdparams\")          "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "可视化观察训练集与验证集的损失变化情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA70AAAF6CAYAAADVrz+JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3Xl8nGW9///XZ/bMJOmWdEv3fW+BtEBZCh7KjgoIPxRlOSDiiso5B0VRAQEXRBDQI18XDoogyCbKLlCgQKGlpSul+0b3Lctk9uv3x6SxLV2SNpk7y/v5eOTR5J77vudNQjt5z3Xf12XOOURERERERETaI5/XAURERERERERaikqviIiIiIiItFsqvSIiIiIiItJuqfSKiIiIiIhIu6XSKyIiIiIiIu2WSq+IiIiIiIi0Wyq9IiIisgczO93MFpvZUjP7zj4e729m/zKzuWb2qpn18SKniIhIY5jW6RUREZFdzMwPfAhMBdYC7wKfdc4t3G2fR4F/OOf+z8w+AVzunPuCJ4FFREQOQiO9IiIisrtJwFLn3HLnXAp4GPjUXvuMAl6u//yVfTwuIiLSaqj0ioiIyO4qgDW7fb22ftvu3gfOq//8XKDEzLrt62RmdpWZzaz/uKrZ04qIiBxEwOsALaWsrMwNGDDA6xgiItIOzJo1a4tzrtzrHK3IfwH3mNllwGvAOiC7rx2dc/cB90H+tbmysvK3hQopIiLtV1Nem9tt6R0wYAAzZ870OoaIiLQDZrbK6wwFtA7ou9vXfeq3NXDOfUT9SK+ZFQPnO+d2HOzEem0WEZHm0pTXZl3eLCIiIrt7FxhqZgPNLARcBPx99x3MrMzMdv0O8V3gDwXOKCIi0mgqvSIiItLAOZcBvgY8DywCHnHOLTCzm8zsk/W7nQQsNrMPgR7ALZ6EFRERaYR2e3mziIiIHBrn3DPAM3tt+8Fun/8N+Fuhc4mIiBwKjfSKiIiIiIhIu6XSKyIiIiIiIu2WLm8WEWmkqqoqNm3aRDqd9jqKNKNgMEj37t0pLS31OoqIiIi0AJVeEZFGqKqqYuPGjVRUVFBUVISZeR1JmoFzjrq6Otaty6/Io+IrIiLS/ujyZhGRRti0aRMVFRVEo1EV3nbEzIhGo1RUVLBp0yav44iIiEgLUOkVEWmEdDpNUVGR1zGkhRQVFemydRERkXZKpVdEpJE0wtt+6WcrIiLSfqn0ioiIiIiISLul0nsAqbfXkP5wi9cxRERarQ8++AAzY+bMmYd1nueeew4zY8sW/ZsrIiLipVxNEpfKeB2jWan0HsCObzzLppH3su2Sx1V+RaRNMrMDfgwYMOCwzj906FDWr1/PhAkTmiewiIiIeCa9fDM1j71L3ZtLvY7SrLRk0QF0e/pz1Nz+JrX3vkPdg/MoungsJd8/keCwMq+jiYg0yvr16xs+f/PNNzn//PN577336NWrFwB+v3+fx6VSKUKh0EHP7/f76dmzZ/OEFREREW/5DXxGZs1WXCaLBfb9e0Jbo5HeA/D3KKbTz0+lx4pvUvztY0n8baFGfkWkTenZs2fDR9euXQEoLy9v2FZeXt6w34033shVV11F165dmTp1KgC3334748aNIxaL0bt3bz7/+c/vsbTP3pc37/r68ccf54wzziAajTJkyBD+8pe/NDn7G2+8wfHHH08kEqFr165ccsklbN26teHxVatW8elPf5pu3bpRVFTEkCFDuOuuuxoe/9vf/sb48eOJRqN06dKFY489lvnz5zf9mygiItKOuVyOzIYdAAT7lxH9xCjI5Mis2+FxsuZTkNJrZn3N7BUzW2hmC8zsmn3sc7GZzTWzeWb2ppmN3+2xlfXb55jZ4d04dghUfkWkI/jFL37BgAEDmDFjBr/97W+B/OXRd955J/Pnz+fRRx/lww8/5Atf+MJBz3XdddfxxS9+kblz5/LpT3+ayy67jJUrVzY6y5o1azjttNMYMmQIs2bN4oknnuDdd9/loosuatjni1/8IslkkpdffplFixbx29/+tmEEe/Xq1Vx00UX853/+JwsWLGD69Ol85Stf2e/ItoiISEfkUhniLy4g/tx8clV1APh7dYJQgMyq9tNzCnV5cwa41jn3npmVALPM7EXn3MLd9lkBTHHObTezM4D7gKN3e/xk55yn3/ld5bf4vybvednz58dR+v0TCQzt5mU8ESmwHd98lvScDQV9zuCEnnS+84wWOfcJJ5zA9ddfv8e2a6+9tuHzgQMHctdddzF58mS2bt1Kt277/zfvW9/6Fueddx4At956K3fffTfTpk1r9D3Ev/rVr+jRowe/+93vCATyL1X3338/xxxzDO+88w6TJk1i1apVXH755Ywfn3+PdPdzr1u3jlwux4UXXthQhEeNGtWo5xYREekIcjUJ4i8uILezjshxQ/CVFgFgPh9FU4bj71TkccLmU5CRXufceufce/WfVwOLgIq99nnTObe9/su3gT6FyHYo9hj5/dYxJB5dwMYR97Dt0ifILNl68BOIiLRCkyZN+ti2l156ialTp9K3b19KSko45ZRTgPylxQey+8RWoVCIsrIyNm7c2OgsCxYsYPLkyQ2Fd1e+SCTCggULAPj2t7/NDTfcwLHHHst3v/tdpk+f3rDvxIkTmTJlCsOHD+f888/n7rvvZt26dY1+fhERkfYsu6Wa2n/MIVebJHrqaEJD95yfI9inK76S9lN6Cz6RlZkNAI4AZhxgtyuAZ3f72gEvmJkDfuucu28/574KuAqgX79+zRH3gPw9iul0+2kU//dx1Px8OrW/fpe6P8/VyK9IB9FSI65eicVie3y9dOlSzj77bK688kpuvPFGunXrxrJlyzjrrLNIpVIHPNfek2CZGblcrlnzfulLX+Kss87iueee45VXXmHq1Kl87nOfaxgdfvnll5kxYwYvvfQSDz/8MNdddx1PPfVUw/3KIiIiHVXmox3g8xE7ayz+LrF97pNeuQWXTBMa3qvA6ZpfQSeyMrNi4DHgm865qv3sczL50nvdbpuPd84dCZwBfNXMTtzXsc65+5xzlc65yl2TsxTCrvKrkV8RaU9mzJhBOp3mzjvvZPLkyQwfPpwNGwpzOffo0aN58803yWT+vU7gO++8QyKRYMyYMQ3b+vTpw5VXXsmDDz7Ir3/9a/7whz+QTCaBfNE+5phj+P73v8/06dOZNGkS999/f0Hyi4iItEa5eP5N69DYPhR/6oj9Fl6A9IrNJGevwuVcoeK1mIKVXjMLki+8DzrnHt/PPuOA3wGfcs41tEXn3Lr6PzcBTwAfvwavFVD5FZH2ZNiwYeRyOX75y1+yYsUKHnvsMW677baCPPc111zDxo0bufLKK1mwYAHTpk3j8ssv55RTTmHixIkAXH311Tz33HMsW7aM+fPn8+STTzJ48GDC4TCvvvoqt956K++88w6rV6/mhRdeYOHChbqvV0REOiTnHIl3llP75CxyNQnMDAsHD3hMsH8Zri5NdvM+xyrblELN3mzA74FFzrk79rNPP+Bx4AvOuQ932x6rn/wKM4sBpwKtes0JlV8RaQ8mTpzIHXfcwV133cWoUaO4++67+eUvf1mQ5+7Tpw/PP/88S5Ys4aijjuLcc8+lsrKShx9+uGGfbDbL17/+dcaMGcOUKVPIZrM8/fTTAHTp0oXXXnuNc845h6FDh3LVVVdxxRVXcN111+3vKUVERNoll8lS98oiUgvWERzUHYuGG3VcoG+X/Jq9q9p+fzHnWn642syOB14H5gG7buq6HugH4Jz7XzP7HXA+sGt2lIxzrtLMBpEf3YX8Pch/cc7dcrDnrKysdLvWjfRadmNNwz2/LpnVPb8ibdCiRYsYOXKk1zGkBR3oZ2xms5xzlQWO1O60ptdmEZGOIFeXIv7SQnJbqglPGkR4dMXBD9pN/KUFZLfXUvyZieTHMVuPprw2F2QiK+fcG8ABv0vOuSuBK/exfTkw/uNHtB2a8EpERERERAot9f4acttrKfrESIL9y5p8fKB/N3JVdbhEGisKHfyAVqqgE1l1dLrsWUREREREWtquyafClQOInTX+kAovQHBID4rPq8TXhgsvqPR6oqH8Lr+G4m/Wl9+R97D9sifILFX5FRERERGRQ5Natonaf8zBpTJYwI+/W/Ehn2vXJc0u27zLDhaaSq+H/D1L6PSL+vJ7zTHUPZIf+VX5FRERERGRpnDOkZyzmsRri7GgH5pp7qbMuu1UP/Q22R3xZjmfF1R6WwGVXxEREREROVQumyPxxhKSs1cRHNyd6KljDrokUWP5OkchnSWzakuznM8LKr2tiMqviIiIiIg0VeLd5aSXbiQ0oR+RE4Zh/uareb5YGH95Cek2vHSRSm8rpPIrIiIiIiKNFR7bl6Ipw4kc0b9FlhYK9C8jt7WGXHWi2c9dCCq9rZjKr4iIiIiI7Et2SzV105fgcg5fLExwUPcWe65g//wSq+nVbbODqPS2ASq/IiIiIiKyS3r1VmqfnUvmo+24ulSLP5+vtIhw5UACvTq3+HO1BJXeNmTv8hv/q8qviIiIiEhHklq4jrp/LcTXOUrs7An4YuGCPG94bB/8XWMFea7mptLbBu0qvz1XXEPsG0f/u/xe/qTKr4i0SpdddhmnnHKK1zFERETatMTsVSRmLCfQrxuxM8bhKwoV7Lmdc2TW7yCzcWfBnrO5qPS2Yf6eJXS+4/R/l9+H56v8isjHXHbZZZgZZkYwGKSsrIzjjz+en/3sZ9TW1nodT0RERBopUNGF0Jg+FJ08Egv4C/78iTeXkpyzuuDPe7hUetsBlV8ROZgTTjiB9evXs2rVKl555RUuvvhi7rnnHo488kg2btzodTwRERHZj1w8RWrxegAC3UuJTByI+Zp/huaDMTMC/buRXb8Tl0wX/PkPh0pvO6LyKyL7EwqF6NmzJ71792bs2LF8+ctf5q233mLz5s185zvf2WPfu+++mxEjRhCJRBg6dCi33HILmUwGgO9973sMHz78Y+f/8pe/zPHHH9/oPM45br/9dgYNGkQoFGLw4MHceeede+zz1FNPccQRRxCNRuncuTOTJk1i9uzZAKTTab797W/Tp08fwuEwvXr14qKLLmrqt0VERKRVy+6opfafc0i8s5xcbdLrOAT7l4FzpNds8zpKkwS8DiDNb1f5Lfmf46j+2XRqfzOT+J/eJ/qF8ZR87wQCQ7p5HVGk3ah9du7HtgUHlBEa2RuXyRJ/ccHHHx/Sg9DQHuQSaepeWfSxx0PDexEcVE6uJknd64v3eCx2xrhmy15RUcHFF1/MAw88wO9//3t8Ph8/+tGP+OMf/8idd97JhAkTWLRoEVdffTWJRIKbb76ZSy+9lFtvvZUZM2Zw9NFHA5BMJvnrX//KT37yk0Y/969//WtuuOEG7rrrLk4++WT+9a9/8c1vfpOSkhKuuOIKNmzYwAUXXMCPf/xjLrjgAhKJBLNnzyYQyL9s3X333TzyyCP8+c9/ZtCgQWzcuJHp06c32/dGRETEa5mPdhB/ZSHm9+Xv3y3QhFUH4isrxqIhMiu3EBrSw+s4jabS246p/IrIwYwePZqqqiq2bNlCcXExP/vZz3j88cc5/fTTARg4cCA//vGP+cY3vsHNN9/MsGHDOProo3nggQcaSu/TTz9NXV0dF154YaOf9yc/+Qlf//rXueqqqwAYOnQoixcv5pZbbuGKK65g/fr1pNNpLrzwQgYMGADAyJEjG45ftWoVw4YNY8qUKZgZ/fr1Y+LEic30XREREfFWaulGEtOX4CstIjp1NL7iiNeRgPwlzsH+ZaRXbcHlnCeXWR8Kld4OQOVXpOUcaOTVAv4DPu6LBA/8eHG4WUd298U5B+RfxBYsWEBdXR3nn38+Zv9+EctmsyQSCTZv3kx5eTmXXnopN9xwA3feeSfBYJAHHniAT37yk3Tu3Li1+6qqqli7di0nnnjiHtunTJnCXXfdRTweZ9y4cZx22mmMGTOGqVOnctJJJ3HeeefRt29fAC6//HKmTp3KkCFDmDp1KlOnTuWcc84hFCrcLJYiIiItxoG/RynRk0dh4dZV2cJH9ic8aVCbKbyge3o7FN3zKyJ7W7BgAZ06daJbt27kcjkAHn30UebMmdPwMW/ePJYsWULXrl0BuOiii6iuruaf//wnmzdv5rnnnuPSSy9t1lx+v59nn32Wl19+mYkTJ/LYY48xbNgw/vGPfwAwYcIEVqxYwe23304oFOKaa65hwoQJVFVVNWsOERGRQnHZHJlN+dex0NAeRE8b2+oKL4CFAm2q8IJKb4fUUH6XX0Ps63uV32Vt66Z0ETl069at48EHH+S8887D5/MxevRoIpEIy5cvZ8iQIR/78PvzSyN06dKFc845hz/96U889NBDdO3aldNOO63Rz1taWkqfPn147bXX9tg+bdo0Bg4cSDQaBfKjz5MmTeL666/ntddeY8qUKfzxj39s2L+4uJhzzz2XX/3qV8ycOZNFixYxbdq0ZvjOiIiIFJZLpom/MJ/4c/PIxfMTVu1+1VVrk16xmZon38Nlc15HaZTW99aBFIy/Vwmdf7nbZc//W3/Z8yXjKfneiQQGd/U6oog0k1QqxYYNG8jlcmzdupU33niD2267je7du3PbbbcB+RJ5/fXXc/3112NmnHLKKWQyGebNm8fs2bP56U9/2nC+Sy65hAsuuIBFixZx8cUXNxTixvrud7/Ltddey9ChQznppJN4+eWX+c1vfsO9994LwJtvvsm//vUvTj31VHr16sWSJUuYO3cuV1xxBQA///nP6d27NxMmTCAajfLQQw/h9/sZNmxYM33HRERECiNXnSD+4nxy1Qkixw/DF/V+wqqDsaCf3PZaMh/tINi39XcGlV7Zd/l9QOVXpD15/fXX6dWrF36/n06dOjFy5Ei+9rWv8dWvfpVYLNaw3w033ECvXr245557uPbaaykqKmLYsGFcdtlle5zvjDPOoFOnTixatIiHHnqoyXm+/OUvU1tby6233spXvvIV+vbty09+8pOGUtupUyfeeust7r33XrZv307Pnj25+OKLueGGG4D8aPEdd9zBkiVLyOVyjBw5kscee2yfyymJiIi0VtnN1cRfWoDLOaKnjSHQs3HzY3jN36szBP1kVm1pE6XXdk1i0t5UVla6mTNneh2jTcqur24ov6SzKr8iwKJFi/aYPVjanwP9jM1slnOussCR2h29NouI7Cnx7grSK7cQnToaf+eo13GapG7aB2TWbaf4omM8uce3Ka/NuqdXPmbXyG/DPb8PzWfj8LvZ/p+651dERERE5HA458glUgCEjxpA7JwJba7wAgT6l+GSGbIbd3od5aBUemW/VH5FRERERJqPyzmSM5ZT+9RscnUpzGf4IkGvYx2SQEUXgoO7Y6HWf8esSq8clMqviIiIiMjhceksdS8vJLXoI4IDy7Fw2yy7u1jQT9GJw/F3K/Y6ykGp9EqjqfyKiIiIiDRdLp6i9tm5ZNZuI3LMYCKTBrW5tW73J7ujllxNwusYB6TSK022R/n92iSVX+kw2uvEf6KfrYiItKzkrJXkdsYp+o9RhEb29jpOs3GpDLVPzia1aL3XUQ5IpVcOmb9XCZ3vPOPj5feKp8gsV/mV9iUYDFJXV+d1DGkhdXV1BINt+zIzERFpfXa9qRo5ehCxs8YT7NvN40TNy0IB/L07k161pVW/gazSK4ftY+X3wblsHKbyK+1L9+7dWbduHfF4vFX/oy5N45wjHo+zbt06unfv7nUcERFpR1JLNhB/di4uk82Xw66t/97XQxHs3w1XnSC3Pe51lP1q/VNtSZuxq/yW/M9xDev8xv9vDtFLJ1DyvRMIDNI6v9J2lZaWAvDRRx+RTqc9TiPNKRgM0qNHj4afsYiIyOFwzpGcvYrU+2vw9+4Mufb9ZnmgXzd4cynpVVvwd415HWefVHql2fl7l6r8SrtUWlqqYiQiIiL75bI5Em98SHr5ZoLDehI5djDma98X1/qKQvh7lJJZsw2O6O91nH0qyE/AzPqa2StmttDMFpjZNfvYx8zsV2a21MzmmtmRuz12qZktqf+4tBCZ5fDtKr+67FlEpG0xs9PNbHH9a/J39vF4v/rX9dn1r9lnepFTRKS1Sby1lPTyzYSPGkBk8pB2X3h3iRw3lNjpY72OsV+F+ilkgGudc6OAY4CvmtmovfY5Axha/3EV8BsAM+sK/BA4GpgE/NDMuhQotzQDlV8RkbbDzPzAveRfl0cBn93Ha/b3gUecc0cAFwG/LmxKEZHWKTS2L0VTRhAe1xez9rEkUWP4O0WxUOu9iLggpdc5t945917959XAIqBir90+BTzg8t4GOptZL+A04EXn3Dbn3HbgReD0QuSW5rVH+f2qyq+ISCs1CVjqnFvunEsBD5N/jd6dA3Zd698J+KiA+UREWpXMpioSM5bhnMPfqYjgoHKvI3kivXwTddOXeB1jnwo+3m5mA4AjgBl7PVQBrNnt67X12/a3fV/nvsrMZprZzM2bNzdXZGlm/t6ldL5L5VdEpJVqzOvuj4DPm9la4Bng6/s7mV6bRaQ9S6/cQvy5eaTXbMMlO/ZEl7maJOkPN5CrSXod5WMKWnrNrBh4DPimc66quc/vnLvPOVfpnKssL++Y77C0Jfstv196GpfKeB1PRET277PA/c65PsCZwJ/MbJ+/U+i1WUTaI+ccyflrqXtlEf6uMWJnj8cXCXkdy1OB/vk1iDOrt3ic5OMKVnrNLEi+8D7onHt8H7usA/ru9nWf+m372y4eym6taba1Svcov1+qJH7fLGp/P7tZzi0iIk3WmNfdK4BHAJxzbwERoKwg6UREWoHkzJUk311BoH83oqeP7fCFF/L39fo6R0mv2up1lI8p1OzNBvweWOScu2M/u/0duKR+FudjgJ3OufXA88CpZtalfgKrU+u3iUeyW6qpfXp2w70LzcXfu5RO95xJaHJfam57HZfUaK+IiAfeBYaa2UAzC5GfqOrve+2zGvgPADMbSb706tplEekwAr07ExrTh6KTR2IBv9dxWo1A/25kN+4kl2hdl3oXaqT3OOALwCfMbE79x5lmdrWZXV2/zzPAcmAp8P+ArwA457YBN5N/EX4XuKl+m3jE162Y0KgK0ovWk3jjQ1wzLrhtZpT8YArZNVXE75/TbOcVEZHGcc5lgK+Rf4N5EflZmheY2U1m9sn63a4Fvmhm7wMPAZe55nwXVESkFcrFk6SXbQIgUNGFyMSBHWqG5sYI9i/D37MzrpWVXmuvr1GVlZVu5syZXsdot5xzpN5fQ3L2KgL9u1E0ZQTmb573UJxzbD72d+TW19Bjyddb9fTnItIxmNks51yl1znaOr02i0hbld1WS/ylBbhUhuLPTMQXCXodqcNrymtzx1gtWZqdmRGe0I/wpEFkVm0lvaL5rmozM0p/MIXs6p3EH3i/2c4rIiIiItJUmXXbqX3mfXCO2BnjVHgbIZdI4TJZr2M0UOmVwxIeXUH0rPEEB3dv3vOeMZRgZW+qb30dl249f2FEREREpONIfbiB+IsL8BVHiJ09AX+3Yq8jtXrZ7bXUPDyDzOrWM6GVSq8ctkD3UsyM7PZaal+Y3yw3rpsZJT+cQnbFDuJ/ntsMKUVEREREmsals/h7dyZ25jh8sbDXcdoEX+coFgm2qlmcVXql2bjaJNkNO4g/O5dcPHXY54ucNYzgkb2ovuW1VnV5hIiIiIi0Xy6TI7ulBoDQqN5ETxmtOWaawMwI9C8js3Zbq/kdXqVXmk2gT1eiU8eQq0lS+8z75KoTh3W+hpmcl22n7i/zmimliIiIiMi+5RJp4i/Mo/a5ueQSacwM82mG5qYK9u8GmRyZdTu8jgKo9EozC/TqTPT0MbhkplmKb+STwwmO70HVjzXaKyIiIiItJ1dVR/yf75PdUk3R5KGasOow+Ht2glCAzKotXkcBVHqlBQTKS4mdMQ5/j05YUeiwztUw2rtkG3V/XdBMCUVERERE/i2zsYraf8zBJdNETxtLcFC515HaNPP5iJ40gvCR/b2OAqj0Sgvxd40RPWkEFvDhkmmym6sP+VyRT48gMLY71TdPw2VzzZhSRERERATSyzZioQDRsyYQ6NHJ6zjtQqCiC77iiNcxAJVeKYDEjOXUPjeXzEfbD+l48/kovWEKmcVbqXtEo70iIiIicvicc7hkftWRyNGDiZ49AX+nIo9TtS/plZtJfbDe6xgqvdLywhMH4iuJEH9xwSFPXR45fySB0eX50d6cRntFRERE5NC5nCPx1lJqnp6DS2Ywv0/38LaA9IotJOeswuWcpzlUeqXF+YpC+Xt8uxVT98pC0ss2Nfkc5vNRcsMUMou2UPe3hS2QUkREREQ6ApfOEH9pAenFGwgOLIeQ3+tI7VawfxmuLk12U5WnOVR6pSAsHCR62hj8PTqRmL0Kl2n6aG3RZ0YRGFFG9c2vabRXRERERJosV5uk9pm5ZD/aTmTyECJHDcBMSxK1lEDfLuAzz2dxVumVgrFggOjU0cROH5uf4Mo17TIH8/soueFEMvM3kXjigxZKKSIiIiLtVWLGMnLVCaJTxxAa3svrOO2eBQMEKrqQXrW1yb/7NyeVXikoC/jxFUdwzpGYvoTEeyub9Beg6P8bQ2BYN6pu0r29IiIiItI4u37fjEweQuzM8QQqunicqOMI9O+GBf24RNqzDCq94g0HmJF6fw3JGcsbXXzN76Pk+yeSmbuRxN8Xt2xGEREREWnzUovXE39xAS6XwxcJ4e8a8zpShxIc0oPic4/CVxTyLINKr3jCfEZk8hBCoytILfqIxPQljZ7VreizY/AP6Ur1TdM8vUxCRERERFov5xyJd1eQeHNpfkNWvzd6Ydc90y7r3VWaKr3iGTMjPHEg4Qn9SC/ZSGL6h407LuCn5HsnkJ69gcQ/GneMiIiIiHQcLpOl7tUPSM1fS3B4T6KnjMaCmqXZK5l126n+y9tkd8Q9eX6VXvGUmRE+oj/hSYMIDure6OOiF4/DP6gL1Te+qtFeEREREdlD3esfklm5hXDlQCLHDsF8mqHZS74uUchkPZvFWaVXWoXw6IqGCQXSq7biUpkD7m9BPyXXn0B61nqSzywpREQRERERaSPC4/tSdNIYr/HNAAAgAElEQVQIwmP7aEmiVsAXDeMvLyG9aqs3z+/Js4rsR64mSd2ri6h9fh65g8zwFr1kPP4BnfMzOWu0V0RERKRDy2zcSWL2KgD8XYsJDiz3OJHsLtC/jNzWGnLViYI/t0qvtCq+4jBFJ48kt72W+LNzycVT+923YbT3nXUkn19awJQiIiIi0pqkl28m/vw8Mss3H/SKQfFGsH83ANIeXOKs0iutTrBfN6KnjCFXkyD+zPvkavb/blD00vH4+3Wi6kaN9oqIiIh0NM45knPXUDftA/xlJUTPGo+FAl7Hkn3wlRYROXYIwX7dCv/cBX9GkUYI9O5M9LSx5JIZMmu27Xc/CwUo+e7xpN9eS/Kl5QVMKCIiIiJeS85YTnLWSgKDyomeNhZfJOh1JDmA0Ihe+EqLCv68Kr3SagW6l1J87lGERvYGwOX2vbZX9PIj8Pcp1UzOIiIiIh2AS2cb5n7x9+xEaFxfik4cjvlVbWTfNPYvrZovGgIgu7WG+MsLKZoygkD30j32sXCA4u8cz86vPUPqlRWEPzHIi6giIiIi0sycc+S21pDdUk12Sw3ZzdXkdsYJje5DZOJAggPKCA4o8zqmtHJ6O0TaBAsHwCw/QcFHOz72eOyKI/D1LqHqxmkepBMRERGRw+WcI7sjTmrJRlJLNzZsj78wn8Rby8is3oovFiY0vh+B/oW/L1TaLo30SpvgK44QO3M88efnEX9pPkUnjdzjJniLBCm57jh2XvMcyWkrCU8Z4F1YEREREWm05IJ1ZFZvJbu1BtJZAHxlxYSG9MDMKPrEKHyxMFYc1pq7ckg00itthi8aInrGOHxdYtS9vIjMhp17PB774lH4ehZTdeOr3gQUERERkX3KJVKk12wjMXsV8RfnU/PErIa5WHI74rhMluDg7kSOH0bs3COJnTWh4dhAz074SiIqvHLINNIrbYovEiR22liS89biLy/Z4zErClLyP8ex89vPk3x9FeET+nuUUkRERKTjcukM2S01+LuXYn4fyTmrSc5e1fC4r3MUf1kJZHMQ8BOZPESFVlqUSq+0ORYKEDlqAJB/1zCzehuhYT0BiH7pKKp/+gbVN00j/OIlHqYUERER6RhytUkyq7bWTzZVTW5nHQDRs8YT6F6Kv3dnwn4f/vJi/N2KseCeFUSFV1qaSq+0aakFH5Gau4ZcbZLwhH74oiGK//s4qv7rBZJvriY8uZ/XEUVERETaBZdz5HbEG8ptcFB3Aj07kauuIzFjGVYUxF9WQnBQOb6yEvxdokB+Gcq9V98QKaSClF4z+wNwNrDJOTdmH4//N3DxbplGAuXOuW1mthKoBrJAxjlXWYjM0jaEj+iPq0uRmrMa0lnCEwcSu7qSml2jvc99weuIIiIiIm2Ocw6yOSzgJ5dIU/fywvxEU5lcfoegP3+Jcs9O+MtLKb5gEhYLadRWWqVCjfTeD9wDPLCvB51zPwd+DmBm5wDfcs5t222Xk51zW1o6pLQ95jMixw3FAn5SC9bh0lkixw6h+L8mU3XdS6RmrCV0dB+vY4qIiIi0arl4qmEEN7ulhtyWagL9yyg6bmh+6UifERraE19ZMf7yEnylRQ0F1/w+rDjs8X+ByP4VpPQ6514zswGN3P2zwEMtl0baGzMjfPQgCPnJrNiCS2WIfWUiNT+bTtWNr1L2zOe9jigiIiLSarhkhuzWalwqQ3BAOQC1/3wfV5MAA1/nGIF+3Qj06QLkf9eKnT7Oy8gih6VV3dNrZlHgdOBru212wAtm5oDfOufuO8DxVwFXAfTrp3s5OxIzI3LkANyYPlgogMvliF07merr/0Xq3XWEJlZ4HVFERETEM+mVm+snm6ohV5WfaMqioYbSGzlmMBby5yeaCvi9jCrS7FpV6QXOAabvdWnz8c65dWbWHXjRzD5wzr22r4PrC/F9AJWVla7l40prY6H8/9KJN5bgH1uC9YpRfdM0uj39OY+TiYiIiLQsl8uR2x5vuEQ5u62W2FnjMJ+P7IYqMht25ieaGtIdf1kJ/rLihmODfbt6mFykZbW20nsRe13a7JxbV//nJjN7ApgE7LP0iuwSqOhC3fJNFP/waKqvfY3Uex8ROrK317FEREREmoVzjlxVHb5YOD+3yZINJN5all/7FiAUwF9WjEtmsKIQ4YkDiRwz2NvQIh5pNaXXzDoBU4DP77YtBvicc9X1n58K3ORRRGlDgoO7Q8BP3auLiH63kuqfvk63v/5/XscSEREROSQulSGzfgfZzfWjuFurIZUleuoYAhVd8HeOERrRC39ZMf6yEqwkssdMyub3eZhexFuFWrLoIeAkoMzM1gI/BIIAzrn/rd/tXOAF51ztbof2AJ6o/wsbAP7inHuuEJml7Qv274ZNHU382XmQypKc/RHhIzTaKyIiIq2bS6bzxXZzNf4epQR6dSZXk6Du5UVghq9rjODAcvxlJfi6xADwl5fgLy/xOLlI61So2Zs/24h97ie/tNHu25YD41smlXQEgd5dKDppJFsvfITc/BrCj2m0V0RERFoP5xxmhsvmqHv9Q7JbqnHViYbHwxP6EejVGV/nGNGzxuPvWowFNGor0hSt5vJmkZYSHFxO9OxRVN80jbp/LSZ0RB/8XWNexxIREZEOxmVz5LbX5kdx69fE9XWJEZ0yAvP7cDUJ/F1j+If1bJhoatckneYzAt1LPf4vEGmbVHqlQyi+5mhqfv0OqcXrSW/YRvTU0QTK9cIhIiIiLcc5h6tJ4iuJABB/di7ZzdUAWDiAr2zPS5JjZ0/wJKdIe6fSKx2Cr2uU4i9VUvPDtym56yTiz80jespoAr06ex1NRERE2qHszjiJGcvJbqqi+PxKfEUhQmP6QM7hLy/BisN7TDQlIi1HNwRIhxH71rGQyJF+fh2+4gjxF+eTXrPV61giIiLSjrhUhsS7K6h98j2ym6oIH9EfC+fHmYIDyggOKse318zKItKyVHqlw/B3ixL72iTq/jSP0ODe+LrE9pgoQkRERORwuGSamsdnkZq/luDg7hSfX0l4dAXm06/cIl7S30DpUIqvPRaLBqn56ZvEzhxPaFQFALm6lMfJREREpK3KxZMAWDhIaEQvYmdPoOj4YfiKQh4nExFQ6ZUOxl8WI/aVidQ9PJ/M0m0AZLfVUvO3mSQXrPM4nYiIiLQluUSauulLqHn0XbLbaoH8EkNaL1ekdVHplQ6n+L8mY5EA1be8BoCvUxGBis4k31lOcs4qnHMeJxQREZHWzOUcqUUfUfPYTNJLNhAa0QtfLOx1LBHZD5Ve6XD83YuJfbmSugfnkVmyFfP7KDppJMEh3UnOXk3y3RUqviIiIrJPLueo/ef7JN5ehr9bjNinjiRy9OCGyapEpPVR6ZUOqfi/JkPIT/WtrwP5Bd8jxw8jOLIXqQXrSC/d5HFCERERaU1yiTSQ/50hOKicopNHED1tLP4uMY+TicjBqPRKh+TvWULs6krif3qfzLL8vb1mRuTowRSdOJzg4O4eJxQREZHWwGVzJOeuoebRd0ivzf/OEB5dQXBAuZYdEmkjVHqlwyr578kQ8DWM9kK++AYHd8d8Ri6eou6tpbhMzsOUIiKFZ2anm9liM1tqZt/Zx+O/NLM59R8fmtkOL3KKtLT0mq3UPDGL5KyVBHp3wd8p6nUkETkEKr3SYfl7lxK76ijiD7xPZsX2jz2e3bCD9Afrib80H5fOepBQRKTwzMwP3AucAYwCPmtmo3bfxzn3LefcBOfcBOBu4PHCJxVpWXWvL6bupYWYz4ieOobof4zCVxLxOpaIHAKVXunQSq47HnxG9W2vf+yx4KDuRE4YRnbDTuLPz8Ml0x4kFBEpuEnAUufccudcCngY+NQB9v8s8FBBkom0MJfO4nL5ySz9vbsQnjiQ2KeOJFDRxeNkInI4VHqlQ/NXlBK78kji988hs+rjV+eFhvSg6OSRZLfWUPvcPHJ1KQ9SiogUVAWwZrev19Zv+xgz6w8MBF7e38nM7Cozm2lmMzdv3tysQUWai3OO9PJN1Dw+k/Ti9QCEBncnPKYP5tevyyJtnf4WS4dX/J3jAaj5yRv7fDzYv4zoKaPBZ6AJK0REdncR8Dfn3H7vAXHO3eecq3TOVZaXlxcwmkjjZLfVEH92LnXTFmNFIXxlxV5HEpFmpgXFpMML9O1E9D+PoPb371F8/QkE+nb6+D4VXYj17oyZ4bI5XF0KX7Hu6xGRdmkd0He3r/vUb9uXi4CvtngikRaSnLeG5KyVWChA5LihBIf20IzMIu2QRnpFgJLvngBAzU/3PdoLNLwIJt5eRu0/5pDdXluQbCIiBfYuMNTMBppZiHyx/fveO5nZCKAL8FaB84kcFpdzuEz+4gR/t2JCI3pTfH4loWE9VXhF2imVXhEg0L8z0csmUPv/3iO7ruqA+4ZGV4AZ8Wfnkt1cXaCEIiKF4ZzLAF8DngcWAY845xaY2U1m9snddr0IeNg557zIKXIoMht3Uvv0bJLvrQIg0LsLkWMGY+Ggx8lEpCWp9IrUK/nuCZBzVP9s+gH383eOEjtzPAQD1D43j8wGLU8pIu2Lc+4Z59ww59xg59wt9dt+4Jz7+277/Mg597E1fEVao1xtkrppHxB/Zi4ukcbfvcTrSCJSQCq9IvUCA7sQvWQ8tffNIrv+wCO4vpIIsTPH4YuFqHt1ccNlUiIiItK6pFduyc/KvHILoXF9KT6vkuAATaom0pGo9IrspuT6EyCdPehoL4AvFiZ65niip4zCAv4CpBMREZHGcpkcAL7OUQIVXSg+9ygiRw3AgnrNFuloVHpFdhMY3JXo58dR+78zyW44+P26vkgQf1n+Eqnk3DWklmxs6YgiIiJyALmqOuIvLaDutQ+A/G1J0U+Mwlda5HEyEfGKSq/IXkq+dyKkstTc/majj3E5R+ajHSTeXEKuOtGC6URERGRfXDpLYtZKap6YRWb9TvzlJWieNREBlV6RjwkM7UbR58ZS+5uZZDfVNOoY8xlFJwwDn5GYuaKFE4qIiMjuspurqXl8Jqm5awgOLKf4/KMIj+2rJYhEBFDpFdmnku+fiKtLU/OLxi8/6YuFCY/tQ2blFjIbd7ZgOhEREQFwufx9u1YSwdcpSvTMcRSdOBxfNOxxMhFpTVR6RfYhOLyMoovGUHvvO2S31Db6uNCYPlg0ROKd5bqkSkREpIW4ZJq6t5fmlyDKOXyRILHTxxLo0cnraCLSCqn0iuxHyQ1TcPE0NXc0frTXAn6Kjh9G0TFDdEmViIhIM3M5R2rxemoem0n6g/X4uhVD/WiviMj+BLwOINJaBUeWU3ThaGrvfofiayfj7xZt1HGBii4NnzvnVH5FRESaQa4mSfzlheS21uDvUUrkmMH4uxZ7HUtE2gCN9IocQMn3T8TVpKi98+0mHedyjrrpS0jNWd1CyURERDoGl8vfLmRFQSzkp2jKcKJnjFPhFZFGK0jpNbM/mNkmM5u/n8dPMrOdZjan/uMHuz12upktNrOlZvadQuQV2SU4pgeRz4yi5lczyG2va/Rx5jNcOkty3lpyNVrCSEREpKlcNkdy3lpqn5yFS2cxv4/Y6eMIDuquq6hEpEkKNdJ7P3D6QfZ53Tk3of7jJgAz8wP3AmcAo4DPmtmoFk0qspfSG07EVSWpaeJob6RyAACJWSubP5SIiEg7llm3ndqn3iM5cwW+0iJcOut1JBFpwwpSep1zrwHbDuHQScBS59xy51wKeBj4VLOGEzmI4LieRM4dQc1db5Pb0fjRXl9xhNDoCjLLN5PZVNWCCUVERNoHl84S/9dC4i/Mh5yj6JTRRE8ZjS8a8jqaiLRhreme3mPN7H0ze9bMRtdvqwDW7LbP2vpt+2RmV5nZTDObuXnz5pbMKh1MyQ+m4HYmqfnVjCYdFx7XFysKktRor4iIyH41LPMX8IFzhI8aQOzcowj27eptMBFpF1pL6X0P6O+cGw/cDTx5KCdxzt3nnKt0zlWWl5c3a0Dp2EITehH55HBqfvk2uarG36NrQT9FU0ZQdMKwFkwnIiLSNjnnSK/cTO0Ts8jVJDEziv5jVP5NY39r+TVVRNq6VvGviXOuyjlXU//5M0DQzMqAdUDf3XbtU79NpOBKfjAFtyNB7d3vNOm4QK/O+IojOOcaZqAUERHp6LLba4k/P4+6Vz4Anw+XygBokioRaXatovSaWU+r/xfOzCaRz7UVeBcYamYDzSwEXAT83buk0pGFjupN5OxhVN/xFrnqZJOOdZkc8Rfmk5q75uA7i4iItGPOORLvLKf2qffIbq0lcsxgYp88An/XmNfRRKSdKtSSRQ8BbwHDzWytmV1hZleb2dX1u3wGmG9m7wO/Ai5yeRnga8DzwCLgEefcgkJkFtmXkh9MwW2ro/bepo32WsCHBf0k560hF29aYRYREWkPdt23a5Zf1i84rCfF51cSGtkb82l0V0RaTqAQT+Kc++xBHr8HuGc/jz0DPNMSuUSaKjSxgvAZQ6i5/U1iX5uErzjc6GMjEwdSs2YbyVkrKTpheAumFBERaV0ym6tIzlhO5JjB+MtKiEweosuYRaRgWsXlzSJtSekPppDbWkftb2Y26ThfSRGhURWkl24iu6W6hdKJiIi0Hrm6FHVvfEj8H++Tq0niEmlA9+2KSGGp9Io0UeiYvoRPHUzNz6eTq0016djw+L5YJEhS9/aKiEg7l/rgI2oem0l62SZCY/pQfP5RBPpoCSIRKTyVXpFDUPLDKeQ2x6n936aN9looQPSU0bq8WURE2j2XyODvXkrs00cSmTgQCxbkrjoRkY9R6RU5BOHJ/Qj/x8D8aG+8aaO9/vISLOjH5XK4bK6FEoqIiBRWrjpB/OWFpFduASA0ri/RqaPxd4p6nExEOjqVXpFDVPLDk8htrCV+36wmH+uSGWqffI/UAi07LSIibZvLZEnMXkXNE7PIrNv+7/t2faZ7d0WkVVDpFTlE4RP6EzppANU/nY6rSzfpWAsH8JUWkZy7pskjxSIiIq1Feu02ap6YRWrOagL9ulJ8biWhEb28jiUisgeVXpHDUPrDKeQ21FD7u/eafGx44kDI5EjOXtUCyUSkI7O8L5rZy2Y2t37biWZ2odfZpJ1JZrCgn+jpY4meNLJJS/mJiBSKSq/IYQifNJDQif2p/skbDZdzNZa/U5TQqN6kP9xAdmtNCyUUkQ7qJuAK4D6gX/22tcB1niWSdsGlMiTeWU6y/vacwKByYp88kkCvzh4nExHZP5VekcNU8oMp5D6qpvYPs5t8bHh8XywcILV4fQskE5EO7DLgbOfcw4Cr37YCGORZImnTnHOklmyk5vGZpBasw9Umgfx6u+bTfbsi0rpp7niRwxT+xEBCx/Wl5rY3iF1xJBZu/F8rCweJnjkeX2lRCyYUkQ7ID+y6hGRX6S3ebZtIo2W31ZB4cynZzdX4y0uInDIaf1mJ17FERBpNI70ih8nMKPnBFLJrq4jfP6fJx/s7RzGf4ZJpLWEkIs3lWeAOMwtD/h5f4GbgaU9TSZvk0llyNUkiJwwjetZ4FV4RaXMaXXrN7GQzG1j/eS8z+z8z+6OZ9Wy5eCJtQ3jqYILH9KH61tdxqUyTj8/VJql5bCapRR+1QDoR6YC+BfQCdgKdyI/w9kf39EojuJwjuXAdiVkrAQj06ETxZyYSGtJDSxCJSJvUlJHeXwPZ+s9/AQSBHPlJMkQ6NDOj9IdTyK7eSfz/3m/y8b5YGH9ZCcn3V5NLaAkjETl09aO6ZcAF5CexOgYY7Jw71zlX7Wk4afUy63dQ+9R7JGcsJ7u1BpfLXx1vAV0cKCJtV1Pu6a1wzq02swBwGvl3jFOAhqZEgPBpQwhO7E31ra8TvWwCFvQ37fhJg8g8OYvk7NUUHTukhVKKSHvnnHNmNg8occ5tAjZ5nUlaP5fOUPfGEjIrt2DFYYo+MZJAv24a2RWRdqEpb9tVmVkPYAqw0Dm3azKMYPPHEml78qO9J5FduYP4n5o+2uvvHCU4ohfpxevJbq9tgYQi0oHMBoZ5HULajuz2ONmNOwlP6EfxuUcR7F+mwisi7UZTRnrvBt4FQsA367cdB3zQ3KFE2qrwmUMJHtWL6lteJ3rJeCzQxNHeCf1JL9tMZtVW/F1iLZRSRDqAV4HnzOx+YA3/nsEZ59wfPMokrVigeynFF0zC/LqMWUTan0aXXufcT83sCSDrnFtWv3kdcGWLJBNpg3bN5LztUw8Tf3AesUsnNOl4XyRI8blH4ouGWyihiHQQx5Ffl3fKXtsdoNIre0iv2UagoosKr4i0W01ap9c59+Guz83sZCDnnJvW7KlE2rDIOcMJTuhJ9Y9fI3rx2CaP9u4qvLmqOqw4jPn0S4iINI1z7mSvM0jbkFm/g7qXFhA5ZjChkb29jiMi0iKasmTRNDM7rv7z64CHgb+Y2fUtFU6kLWpYt3fpNuoenn9I58juiFPzxCzSH6xv5nQi0lGYWRczu8TMvlv/ZxevM0nr4nKOxDvLsViY4NAeXscREWkxTRlCGgO8Xf/5F4GTyS+DcHVzhxJp6yKfGk5gXA+qf/waLptr8vG+TkX4e3YiMXs1LplugYQi0p6Z2bHAMvKv0eOALwHL6reLAJBeupHctloilQOafFWSiEhb0pTS6wOcmQ0GzDm30Dm3BtA7xyJ7MZ+P0htOJLN4K3WPLGj68WZEJg6CdIbknNUtkFBE2rk7ga845yY75z7rnDsO+DLwK49zSSvh0hmS763EX15CYGC513FERFpUU0rvG8A9wO3AEwD1BXhLC+QSafMi540kMKY71TdPO6TRXn/XGMFhPUktWk92Z7wFEopIOzYMeGSvbX8DtAi4AJCLp7BIiPCkQVqaSETavaaU3suAHcBc4Ef120YAdzVvJJH2wXw+Sm44kcyiLdT9beEhnSN8RH8s5Ce7saqZ04lIO7cEuGivbReQv+RZBH+nKLFPHUGge6nXUUREWlxTlizaCly/17Z/NnsikXak6PxRVI8so/rmaRRdMKrJMzH7ikL5dRODutdKRJrkm8A/zOwbwCpgADAUONvLUNI6pJdtItCnKxZu0iIeIiJtVlNmbw6a2Y1mttzMEvV/3mhmoZYMKNKWmd9HyQ1TyCzYTOLxRYd2jvrCm9lYhcu55ownIu2Uc+5NYDD525JmAXcDQ+q3SweW2VxF3WuLSS1c53UUEZGCacqw08+AU8jPBDm+/s9PAD9tgVwi7UbRhaMJDO9G1c2v4XJNv7cX8oU3/sz7pD/c0MzpRKQ9MrMKAOfcn51zP3PO/Zn8ZJRaiLUDc86RnLEcKwoSGlPhdRwRkYJpSum9APikc+4F59xi59wLwLnAhS0TTaR9ML+Pku+fSGbuRhJPLT6kc/i7l+Dv2YnkeytxyUwzJxSRduhJoM9e2/pQPxGldEyZFVvIbq4mfOQALKhLm0Wk42hK6d3f1H6a8k/kIIouGoN/aFeqb5qGc02/RHnXEkYumSE5V0sYichBDXPOzdt9Q/3XIzzKIx5zmRyJWSvwdY0RHNLD6zgiIgXVlNL7KPC0mZ1mZiPN7HTy7yQ/2jLRRNoPC/gp+d6JpOdsIPH0IY72lhUTHNqD1MKPyFXVNXNCEWlnNpvZHssT1X+91aM84jGXzuDvHCMyaRDm03iFiHQsTSm9/wO8BNzLvyfFeAX47xbIJdLuRC8ei39QF6pvPLTRXiB/SVpRiOxOlV4ROaA/AI+Z2dlmNsrMzgEeA37ncS7xiK8oRHTqaAK9OnsdRUSk4A54Q4eZfWKvTa/Wfxiw67f244GXD3KeP5BfJmGT+//bu/Mwuao6/+Pv762116STdHeSztLpbJBAdgJIWNzYRuFxGYVBBR8VdFAG/TmD4rC44DYzCqPoCIjKDCji6IiCIC6gsmWBkJB9T7qTdGdPr7We3x9VaTp7Ol3pW1X9eT1PPem699apTzVFn/rWufcc5844wv5rgFuy7bYCn3DOvZbdtzG7LQUknXNzjvVcIvkqM9p7Pns/8jixJ9cQ/btJvW7DKw1T/t6z9C29iBzP14EE8O/AaGAzmYL3236GEn/EV20jOGIwXmWJ31FERHxxvFkMfniU7QcK3gPFb8Nx2vkxmWUTHjrK/g3Ahc65PWZ2GXAfcHaP/W92zu08znOI5L3SD06n9St/Yf8XnyVy+UTMel+8mmc450hu2klwzDAVwCJyJBcCv3DO/ZuZjSCz0sIZQA2gaeAHkNTuNrpeWEt4ah3Rucf7uCYiUpyOeXqzc27cUW4N2ds459xx/4I65/4C7D7G/hecc3uyd1/i8BknRYqChQJU3Ho+iQVbiT219qTbSTbuofPPK0msbc5hOhEpIt8jc4YUwH+Q+ZI7TeZL5eMys0vNbJWZrTWzzx3lmPeZ2XIzW2Zmj+QkteSUc46u+euxSJDI9NF+xxER8U1vruntLx8BftfjvgN+b2aLzOx6nzKJ5Ezph6YTGDOI/V989qSv7Q2OqiJQU5lZwiiuJYxE5DB1zrnNZhYELgWuBz4BvOl4DzSzAJn5Oy4DpgBXm9mUQ46ZCHweOM85NxW4Ocf5JQeSW3aT2raPyIyxWCTkdxwREd/kVdFrZm8mU/Te0mPzPOfcLDKd741mdsExHn+9mS00s4U7duw4xWlFTo6Fg5nR3pebiD2z7uTaMCM6twHXmSC2ZEuOE4pIEdhvZrVkTnNe5pxry24/kcpnLrDWObfeORcHfgZcecgxHwPuPXCWlnOuJUe5JUdcKk1swQa8QSWEThvudxwREV/lTdFrZtPITLJxpXOue0kF51xT9t8W4FdkOuMjcs7d55yb45ybU11dfaoji5y00utmEBhd2aeZnAPVFYTG1xBf3kS6tSvHCUWkwH0HWAA8TGbUFuA8YOUJPLYO6PltWmN2W0+TgElm9ryZvZRdxvCI9IW0T9KOQN1gInMbMC9vPu6JiPgiL/4KmtkY4JfAB51zq3tsLzOzigM/A7G21x4AACAASURBVBcDr/uTUiR3LBKk/HPziL+whdifNpx0O5HZ9XiDSkl3xXOYTkQKnXPuG8DbyJx+/LPs5ibgozl6iiAwEbgIuBq438yOuBaOvpD2h4UClJwzgdCoIX5HERHxXb8UvWb2U+BFYLKZNZrZR8zs42b28ewhtwNDge+Z2WIzW5jdXgv8zcxeA+YDTzjnnuqPzCKnWtlHZuHVVdDah2t7vbIIZVfMJFhdmeN0IlLonHOrnXPrDrm/9AQe2kRmmaMDRmW39dQIPO6cSzjnNgCryRTBkgdiy5pIbt/ndwwRkbxxvCWLcsI5d/Vx9n+UI3z77JxbD0w/VblE/GSRIBW3zGPfTb8j/txGIheNO7l2zHCJFIk1zYROH3FSyyCJiPSwAJhoZuPIFLtXAf9wyDH/R2aE90dmNozM6c7r+zWlHFFqbwexBesJTRpOcPggv+OIiOSFvDi9WWSgKvvYLLwR5ez/4nN9aie5ZRddL68jsVZzyYhI3zjnksAngaeBFcDPnXPLzOxLZnZF9rCngV1mthz4M/DPPefjEP/EFmyAYIDIzLF+RxERyRsqekV8ZNEQFbfMI/7sRmJ/2XjS7QTHVeMNq8gsYZRIHf8BIiLH4Jx70jk3yTk33jl3V3bb7c65x7M/O+fcZ5xzU5xzZ/a4blh8lGzaQ7JxN5Hpo/FKwn7HERHJGyp6RXxWdv1svNoyWr908qO93UsYdcSJLdUSRiIiA41LO7rmr8fKo4SnHDrZtojIwKaiV8RnVhKi/F/OI/bHDcSe33zS7QRrKwmOqyb+ehPpNi1hJCIysDhCk4YTPbsBC+jjnYhIT/qrKJIHym6Yg1dd2qfRXoDonHqCwwfhUic3G7SIiBQm8zwiU+sIjRnqdxQRkbyjolckD3hlYcr/+Txiv19H/KWTPz3ZK49SevEZBAaV5DCdiIjks9jSRuJrm/2OISKSt1T0iuSJsk/MwRtWyv4+jvYCpNtjdL266aTX/xURkcKQbu0i9spGUlv3+h1FRCRvqegVyRNeeYTy/3cusd+tJT6/sU9tJbfuIb54M8n1O3KUTkRE8lHXwg3gGZHZ9X5HERHJWyp6RfJI2Y1zsSElfR7tDU2oxRtSRteijbikljASESlGyeZ9JDfuJHLGKLyyiN9xRETylopekTziVUSo+My5xJ5YQ3zR1pNux8yInj0e1x4j/npTDhOKiEg+cC67RFFpmPAZo/yOIyKS11T0iuSZsk/NxaqifZ7JOTh8EMGxQ4kt3UK6I5ajdCIiki8i00YTPWcCFgr4HUVEJK+p6BXJM15llPJPn0vX46uIv7qtT21FzxpHqKEGzHKUTkRE8oGZERo7jNBYLVEkInI8KnpF8lD5p+ZigyK0frlvo71eRQkl503EKwnnKJmIiPgttrSR2GLN0C8icqJU9IrkIW9wCeU3n0PXr1aSWLK9z+2ldrbS9fI6fUASESlw6fYYscWbSO3pwHQWj4jICVHRK5Knyv/pHKwywv4v/6XPbaV2tRFfvpXkpp05SCYiIn6JLdoIaUd0zji/o4iIFAwVvSJ5yqsqofyms+n6xXISrzf3qa3QxOF4VWV0LdiAS6ZzlFBERPpTamcriXUthKfW4VVE/Y4jIlIwVPSK5LHyT5+DlYdp/UrfRnvNM6Jzx+HaYsSXawkjEZFC1LVgAxYNEZk22u8oIiIFRUWvSB7zhpRS9qm5dP58GYnlLX1qKziyiuDoIcSWbCHdGc9RQhER6S/RsxqIzpuIhYN+RxERKSgqekXyXPlnzsVKQ30e7QWInNVA5IxRWtNRRKQABYaVExqtJYpERHpLRa9IngsMK6Psxrl0/ux1Eit39K2tQSVEZozBgip6RUQKRez1Rjr/ugqX0pwMIiInQ0WvSAEo/+ybsJIQrXf9NSftJbbsovNvq7WEkYhInkt3xokt3ozrSmABfWwTETkZ+uspUgAC1WWU/eNZdD6ylMTqvi875NpiJNY0k9y8OwfpRETkVIm9ugmSKSJnNfgdRUSkYKnoFSkQ5Z99ExYJ0PbVvo/2hiaPwBtUQmzBep0uJyKSp1J72kms3k7otBEEBpf6HUdEpGCp6BUpEIHacko/PoeO/1lCcl3fRmgzSxg1kG7tIr5ia44SiohILsVe2QShIJEZY/2OIiJS0FT0ihSQin8+D0IBWnMw2hscNYRAXRWx1zaT7krkIJ2IiORS9NzxlF50Gl405HcUEZGCpqJXpIAERlRQdv1sOh56jeSGPX1uLzq3gZJzJmARrfkoIpIvnHM45/BKIwTrqvyOIyJS8FT0ihSYilvOg4DlZLQ3MLiU0PgazCwHyUREJBcSK7fR8dRSXDzpdxQRkaKgolekwARGVlL20Vl0/HgxyU17c9JmfOU2Ov68QksYiYj4zMUSmRmbAUJaU11EJBdU9IoUoIrPzQPPaP1abtbtdak0yY07STb2/ZRpERE5ebHXtuBiSaJzG3QWjohIjqjoFSlAgVGDKPvITDoefJXkln19bi982gi8yuwSRmktYSQi4ofUvk7iK7YSmlhLYGi533FERIqGil6RAlX+uXkAtH39b31uywIekbPGkd7XSWLltj63JyIivRdfugU8j8iser+jiIgUlX4res3sQTNrMbPXj7LfzOw/zWytmS0xs1k99l1rZmuyt2v7K7NIPguOGUzph2fS/sArpBr7PtobHD2EwIjBxBZvxiVSOUgoIiK9ET17PKVvn4pXGvY7iohIUenPkd4fA5ceY/9lwMTs7Xrg+wBmNgS4AzgbmAvcYWaav18EqPj8PEg7Wr/5fJ/bMjOi54yn5K1TME2eIiLSb1za4VJpLBQgOHyQ33FERIpOvxW9zrm/ALuPcciVwEMu4yVgsJmNAC4BnnHO7XbO7QGe4djFs8iAEayvovTa6bTft4jU1v19bi8wuJRgbeYDl2ZyFhHpH4m1zbT9ahHp9pjfUUREilI+XdNbB2zpcb8xu+1o2w9jZteb2UIzW7hjx45TFlQkn1Tcej4k07T+2ws5a7Nr4QY6/7QiZ+2JiMiRuUSS2Csb8aIhTKc1i4icEvlU9PaZc+4+59wc59yc6upqv+OI9ItgwxBKPzid9v9aSGp7a07atEiI5OZdJJu0hJGIyKkUW9KI60wQ0RJFIiKnTD4VvU3A6B73R2W3HW27iGRVfOF8iKdoy9Fob3jKSKwiStf89bi0TnMWETkV0m1dxJc1EWyoJlhT6XccEZGilU9F7+PAh7KzOJ8D7HPObQOeBi42s6rsBFYXZ7eJSFZwwlBKrjmT9u8vINXS1uf2LOARnTOO9N4OEqu35yChiIgcKr4q8/c1Orve3yAiIkWuP5cs+inwIjDZzBrN7CNm9nEz+3j2kCeB9cBa4H7gHwGcc7uBLwMLsrcvZbeJSA8V/3oBLpai7d9zM9obHDuUwPBBxF7bjEunc9KmiIi8ITJrLGV/Nx2vPOp3FBGRohbsrydyzl19nP0OuPEo+x4EHjwVuUSKRWjSMEquPoP2exdQ/s/nEagu61N7Zkb0TRMwz8O8fDopRESksDnnIJ7EIiECQ8v9jiMiUvT0SVakiFR84QJcZ4K2b72Yk/YCg0rxKjIjEC6ZykmbIiIDXXLDDlp/sYDUnna/o4iIDAgqekWKSOj0akrefwbt351PaldHztrt+PMKOp9blbP2REQGKpdM0bVwI155FG9Qqd9xREQGBBW9IkWm4l8vwLXHaf92bkZ7AQJDyjJLGG3bm7M2RUQGoviyJlx7jOjcBszTEkUiIv1BRa9IkQlNraHkvVNo+8+XSe/OzWhveGodVhbREkYiIn2Q7ogTW9JIcMxQgiMG+x1HRGTAUNErUoQqbrsQ1xqn7Z6Xc9KeBQOZJYx2t5NY25yTNkVEBprk5l2QThM9a5zfUUREBhQVvSJFKHRmLdF3n07bPS+R3tuZkzaD44YRqKkkvnxrZuZRERHplfBpIyh/9xy8yhK/o4iIDCgqekWKVMVtF+D2xXI32mtGyQWTKbt8Gma6Dk1E5EQ550i3dgF0z4gvIiL9R0WvSJEKzxhB9MrJtN39Eul9XTlp06uIYuEgLu1wsWRO2hQRKXbJzbtp+98FJLfv8zuKiMiApKJXpIhV3H4hbm8Xbd/JzWgvZEYs2p94jc4X1uSsTRGRYuVSaWIL1uNVlhCoqfQ7jojIgKSiV6SIhWeNJPrOSbR960XSrbGctGlmhEZVkdy4k2SzRi1ERI4lvnIb6dYuIlqiSETENyp6RYpcxe0X4vZ00f7d+TlrM3zmKKw0nFnCSJNaiYgcUborQWzxZgJ1VYRGDfE7jojIgKWiV6TIhefUEbl8Im3/8QLpthyN9gYDROaMI72zjcS6lpy0KSJSbFLN+yGlJYpERPymoldkAKi8/ULSuzpp/96CnLUZaqjGG1aholdE5ChCY4dS8f65BKrK/I4iIjKgqegVGQDCZ48icsl42v79BdLt8Zy0aWaUvnUKpW8/IyftiUj+MLNLzWyVma01s88dYf91ZrbDzBZnbx/1I2c+S+1sBcAiIZ+TiIiIil6RAaLijotI7+ig/b8W5qxNrzSMeYaLJ0l35aaYFhF/mVkAuBe4DJgCXG1mU45w6KPOuRnZ2wP9GjLPJZv20P6bxSQ27PA7ioiIoKJXZMCInDuayNsaaPvm86Q7clegulSatv97hdjL63PWpoj4ai6w1jm33jkXB34GXOlzpoLh0o6u+euxiijBMUP9jiMiIqjoFRlQKu64kHRLOx0/WJSzNi3gERpfQ2L9DpI79uesXRHxTR2wpcf9xuy2Q73HzJaY2S/MbPTRGjOz681soZkt3LGj+Ec+E6u3k97bQXTOOCygj1kiIvlAf41FBpDIvLGE31xP6zefx3UmctfutNFYSYjYy1rCSGSA+A1Q75ybBjwD/ORoBzrn7nPOzXHOzamuru63gH5w8SSxVzcRqK0kOFajvCIi+UJFr8gAU3nHRaS3t9F+fw5He0MBIrPrSe1oJalr2EQKXRPQc+R2VHZbN+fcLufcgTXQHgBm91O2vJba2wFAdG4DZuZzGhEROUBFr8gAE7mwnvCFY2n9xvO4rtyN9oYm1OINKSO5bV/O2hQRXywAJprZODMLA1cBj/c8wMxG9Lh7BbCiH/PlrWBNJeV/P5fAsAq/o4iISA8qekUGoMrbLyS9tZX2H76aszbNjLLLplFy3sSctSki/c85lwQ+CTxNppj9uXNumZl9ycyuyB52k5ktM7PXgJuA6/xJmz+STXtwaYcF9dFKRCTfBP0OICL9L/zmcYTnjaHt63+j7KOzsEhu/hRYONNOurULAh5eaTgn7YpI/3LOPQk8eci223v8/Hng8/2dK18lt++j4/evEz17POEpI/2OIyIih9DXkSIDkJlRcfuFpBr30/Gj3I32QmYil7Zfv0Js0cactisiko+cyy5RVBomNLHW7zgiInIEKnpFBqjI2xoInzuK1q/9DRdP5qxdCwcJTx5BYm0zqZ2tOWtXRCQfJda1kN7VRmR2PRYK+B1HRESOQEWvyABlZlTccRGpzfvo+MlrOW07Mn00Fg3RNV9LGIlI8XKJFLFFG/GGlRMaX+N3HBEROQoVvSIDWOTi8YTm1tH61b/iEqmctWvhIJFZY0k17ye5aWfO2hURySfpjhgWCmiJIhGRPKeJrEQGMDOj8o4L2fV3j7D7vT8nfHYdgfFDCE4YQnB8Fd7gkpNuOzRxOPGV20jv7cxhYhGR/BEYVErZu2ar4BURyXMqekUGuMhlEym9fjZdv1lF1+OrDtrnDS0hMOFAEZz5N3CgIK4uO+YHPfOMsnfMwAIezjl9KBSRopLYsINgXVX3rPUiIpK/9JdaZIAzM6p+8E74wTtJt8dJrd9Dcu1ukut2k1y7m9Ta3cSf30LnT1+H9BvX51pFOFMEHxgZzhbDwQlD8EZWYJ6HBTJXUCRWbSe+ZjvhScMJNVRjIf3pEZHCldrZSuezKwnPGEN05li/44iIyHHok6eIdPPKwnhn1hI68/BlN1wsSXLjXlI9CuLk2t0klzTT9euVkEi/cXA0mCmAswWxN2kQlDq6XlhL1/z1hMZVE5o8nGB1ZT++OhGRvuteoigaIjK1zu84IiJyAvqt6DWzS4F7gADwgHPu64fs/zbw5uzdUqDGOTc4uy8FLM3u2+ycu6J/UovIARYJEpo8jNDkYYftc8kUqS37Sa7LjAxnRoozI8axZ9bhOjNLInnjKwldWIfrSBB7cTNsjmdOlR5fRWjiUILjqrCIvosTkfyV3LSTVPN+oudO0KnNIiIFol/+WptZALgXeDvQCCwws8edc8sPHOOc+3SP4z8FzOzRRKdzbkZ/ZBWR3rNggOC4KoLjquBt4w/a55wjva31jZHhdXtIPbeTVHMriYXbcakU5V9/E52/WULiL1uh02VPm67qcdp05r5XHvHpFYqIgEum6VqwEW9wKaFJw/2OIyIiJ6i/vqKcC6x1zq0HMLOfAVcCy49y/NXAHf2UTUROITMjMLKSwMhKIhfUH7TPOUeyaR/xhRuxshChc0fg2pMkl+6m66m1pO/ff9Dx3vDy7muHD51gy6s6+ZmmRUROhIsn8SqjRM4YhXmanE9EpFD0V9FbB2zpcb8ROPtIB5rZWGAc8Kcem6NmthBIAl93zv3fqQoqIv3HzAiNGkxo1AxcMkViw04Sq7djZUEGf/MyDI/Eqh2kNu4jte6NCba6/rCe9E9eO7itISXdE2kdOsGWV3PsmaZFRE6EVxqm7JIz/Y4hIiK9lI8Xo1wF/MI5l+qxbaxzrsnMGoA/mdlS59y6Qx9oZtcD1wOMGTOmf9KKSE5YMEB4Yi3hibWkW7vwKqIApNpaSVkH4cvGUjZxLl5Z5hTndEec1Ia93adNp7KTa8VfaqTz0WUHzzRdHn5jdLhHMRwYX0VgVCXmeb68ZhEpHPE12wmOGIxXHvU7ioiI9FJ/Fb1NwOge90dltx3JVcCNPTc455qy/643s2fJXO97WNHrnLsPuA9gzpw57tD9IlIYDhS8AKGGGlw8RezVTcQWbyI4agjhKSMJjqzCm1pDaGrNYY938SSpTft6XEec/ff1Frp+sxriPb5TiwQINlQdcfmlwNjBWCjQHy9ZRPJYak87Xc+vIXz6SKJnjz/+A0REJK/0V9G7AJhoZuPIFLtXAf9w6EFmdhpQBbzYY1sV0OGci5nZMOA84Jv9klpEfBeqH0aofhjp/Z3E1zSTWNNMsnk/wZFVuHQa1x4/qEgGsHCQ4MShBCcOPaw9l0qTatx/0OjwgQm2Yn/cgOtIvHFwwAjUDz5sdDg4YQjBhiosGjrVL19EfHZgiSJCQcLTdRaZiEgh6pei1zmXNLNPAk+TWbLoQefcMjP7ErDQOfd49tCrgJ8553qO0p4O/MDM0oBH5preo02AJSJFyqssITq7nsjMsZDKrAmc3LKbzj+tIDByMOFJwwmOGYoFjn2qsgU8gmMHExw7GN7acNA+5xzp7W3dI8OpHksvdTyyFLe3q0dDEKir7DGhVna0OHsKtVehmaZFikGycQ+prXuJzG3A0xddIiIFqd+u6XXOPQk8eci22w+5f+cRHvcCoFkjRAQgM2OqlznlOFBdQWTmGOKrm+l8dmVmLeEJtURmjj2p05LNjMCICgIjKojMG3vY/vTujjdGhg8sv7R2N12PryLd0n7QsV5NWY8i+OAJtrwhJZpYS6QAuHSa2IL1eJUlhE8b4XccERE5Sfk4kZWIyAnxSiNEZowlPG0Mqa17iK/eTrJxN5GzxgGQ2tmKN7gUC+bmulxvSCnhuaWE5446bF+6NUZy3cGjw8m1u4n/eQOdDx0y0/TgaPfocNX/vDtn+UQkx1JpAsMHERw15LhnkYiISP5S0SsiBc88IzhqCMFRQ3CpNGaGS6Xp+P3rOOcINdQQnjScwNDyU5bBq4gQnjECZhw+GuQ6EyQ37OkeGe6eXGvDXhW8InnMQkFK3jTR7xgiItJHKnpFpKh0j8Z4RsmbTye+ejuJNdtJrNyGN6yc6JxxBEcM7t9MJSFCU2oITTl8pmkRyU/xFVvxhpYTrKn0O4qIiPSRil4RKUpmRnDEYIIjBuNiCeLrWkis2g5e5lra9P5O0l0JAtUVur5WRA6S2tdJ1/z1hCbUqugVESkCKnpFpOhZJERkSh3h00d2b4stbyKxYhve4FJCk4cTHl+DRTQzq4hAbOEG8LzMbPEiIlLwVPSKyIDRc0Q3OqueQFUZ8dXbib28ntjCDYQmDqfk3Ak+JhQRvyW37SW5eReR2fV4pWG/44iISA6o6BWRAcnCQcKTRxCePILUrjbiq7d3TyrlnCOxejvBMUPxSvShV2SgcGlH1/z1WHmE8JQ6v+OIiEiOqOgVkQEvMLT8oBHe9J4Oul5YCy+uIzhmCOHJIwiMHKxrf0WKniPUUI1XWYIFtUSRiEixUNErInKIwJAyyt41m8Tq7STWNtOxaRdWHqH0bVMJVJX5HU9EThHzPCJnjvY7hoiI5JiKXhGRIwgMLiUwt4HI7HqSm3eRWL8Dr6IEgMTGnWBGcPQQzNPor0gxiC1rwisJEWrQ0mIiIsVGRa+IyDFYwCM0rprQuOrubfHlTaSa92MlYUITawlPqu0uiEWk8KTbuogt2khw7FAVvSIiRUhFr4hIL5VeOo1k424Sq7cTX7qF+JIthKeNJjq73u9oInISuhZtBND/wyIiRUpFr4hIL5lnhMYMJTRmKOn2GIk1zQSqK4DMiFF8+VZCk4YTGFzqc1IROZ5ky36S63cQnj4arzzqdxwRETkFVPSKiPSBVxYhMmNM9/1USyvx5VuJL2siUFtJaNJwQvXDupdDEpH84ZwjNn89VhLWBFYiIkVMRa+ISA6FGqoJjBhEYm0zidXNdP11NbH56yl/31wVviJ5KDy1DsywkP7/FBEpVip6RURyzMuOGoXPGEWqeR+pXe3dBW/XS+vwqkoJjavGwvoTLOInMztokjoRESlO+sQlInKKmBnB4YMJDh8MgEumSG7fR3rFVrrmryfUUEN4ap2u/T0FnHOYaTkpObrYsiZIpghPG633iohIkVPRKyLSTywYoOzKmaR2tpJYtZ3EuhYSq7dTctFpGm3KsfiSLaSa91PylilY0PM7juSZdEec2CsbCY6sUsErIjIAqOgVEelHZkawupJgdSWROeNIrNxGcGQVAMmmPbh4kuDYYZinD+InK90eI7ZkS6agUcErRxB7ZSOkHdGzxvkdRURE+oGKXhERn3jR0EEzP8dXbSO5aRdWHiUydSShicM1uc5JiC3amClo5qqgkcOldrWRWNNMeGodXmWJ33FERKQf6CtwEZE8UXLR6ZS85XS80jBdL6+n9efzia/c6nesgpLcsZ/EupZMQVOhgkYO17VgPRYJEpk+5vgHi4hIUdBIr4hInjDPCI0dRmjsMJLN+4m/3giBzEivSyRJt8c16dVxxF/bgpWEiEzXmqtyZNHZ40h3xLCIPgKJiAwU+osvIpKHgrWVBGundN+Pr24mNn89wdFDCJ8xikBtpSbgOYKSCyaT3teJhdS9yZEFqisIUOF3DBER6Uf6VCAiUgBCDdW4eJLEiq10/G4J3rAKImfWZSa9UvGLS6XBDAsHCVSroJHDxZY1kd7bQfSc8VhAV3eJiAwk+qsvIlIAvJIw0ZljKX/fXKLnjsfFEsRXbOsueF3a+ZzQX7HXNtP++Cu4RMrvKJKH0l0JYos3kW6PqeAVERmANNIrIlJALBggfNpIQpNG4LoSQGbN0fbfvEpo0nDCp43AKwn7nLJ/pdu6iL/eRHDsUM12LUcUW7wJEiktUSQiMkCp6BURKUDmGVaaLW6TKQLDKogv3kx86RZC42sJn1FHYNDAmPSqa8EGIDNBkcihUns7SKzcRmjScAJVZX7HERERH6joFREpcF5lCaVvnUJqXwfx15tIrGsmsaaZ8vfPLfpR3+T2fSQ37iQyYwxeecTvOJKHYos2QjBAZNZYv6OIiIhPVPSKiBSJwKBSSs6bSGTWWFLb9nYXvF0LNxAYVkFwzFDMK65Jr+KrtmFlEcJnjvI7iuSp6NkNpPZ04EWL+wsgERE5OhW9IiJFxisJ4zXUAOASKRIbdxJf2ohXESV8Rh2hCbVYsDiufS05fzLp1q6ieT2SO85lJnfzyqN45VGf04iIiJ80haGISBGzUIDyd8+h5KLTIBKk68V1tP18Psnte/2O1icukcTFk5hnBAaV+B1H8lBi1XY6nn4dF0/6HUVERHzWbyO9ZnYpcA8QAB5wzn39kP3XAf8GNGU3fdc590B237XAv2a3f8U595N+CS0iUgTMM0LjqgnWDyPVvJ/48ia8wZkJfVI7WyEULLjCMbZ4C4l1zZS/ew4W1klLcjAXTxJ7dRPeoBLQjN4i/Wr//v20tLSQSCT8jiIFLBQKUVNTQ2VlZU7a65dPCmYWAO4F3g40AgvM7HHn3PJDDn3UOffJQx47BLgDmAM4YFH2sXv6IbqISNEwM4LDBxEcPqh7W9f89aSa9xMcM5TwmaMI1uSmczmVUvs6iS9vIjS+RgWvHFHstS24rgTRt0/tXstaRE69/fv309zcTF1dHSUlJfr/T06Kc47Ozk6amjJjobkofPvr9Oa5wFrn3HrnXBz4GXDlCT72EuAZ59zubKH7DHDpKcopIjKglFx0OuFpo0lu30fHE6/R/sRrJLfm93eKsQXrwfOIzKr3O4rkoXRr9kuRCTUEhlX4HUdkQGlpaaGuro7S0lIVvHLSzIzS0lLq6upoaWnJSZv9VfTWAVt63G/MbjvUe8xsiZn9wsxG9/KxmNn1ZrbQzBbu2LEjF7lFRIqaVxomOrueivfNJXp2A+mOGOn9XQC4ZBqXTPmc8GDJpj0kt+wmMn00Xqlm4z1VzOxSM1tlZmvN7HPHOO49ZubMbE5/5juW2JJG8IzI7Hq/o4gMOIlEJnOUtQAAFtxJREFUgpKSwrpcRvJXSUlJzk6Tz6eJrH4D1DvnppEZze31dbvOufucc3Occ3Oqq6tzHlBEpFhZKEB4Sh3l7zmL0KRaABJrttP22AJiizeR7sqPa7OSW3ZjFVHCU4/43afkQI9Lki4DpgBXm9mUIxxXAfwT8HL/Jjy26NxxlL5tKl6p1m0W8YNGeCVXcvle6q+itwkY3eP+KN6YsAoA59wu51wse/cBYPaJPlZERHLDPMO8TNfgDS0nMKyC2Kubafv5fDpfXEt6f6ev+aLnjKfsHdOxQD59Z1t0TvSSpC8D3wC6+jPc0TjncOk0FgoSHDHY7zgiIpJH+utTwwJgopmNM7MwcBXweM8DzGxEj7tXACuyPz8NXGxmVWZWBVyc3SYiIqdQsKaS0rdPpexdswg1VJNYvZ3Ov632JYuLJUi3ZmorL6rTmk+x415WZGazgNHOuSeO11h/XXqUWNdC+/+9Qro9dvyDRURkQOmXaS+dc0kz+ySZYjUAPOicW2ZmXwIWOuceB24ysyuAJLAbuC772N1m9mUyhTPAl5xzu/sjt4iIQGBwGSXzJhGZVY+LZU5zTnfG6XxuFeHTRxIcM+SUn84We3Uz8TXbKf/7uXjR0Cl9Ljk2M/OAb5Htp4/HOXcfcB/AnDlz3KnI5BIpYos2YqVhTNd6i4jIIfrt/DDn3JPOuUnOufHOubuy227PFrw45z7vnJvqnJvunHuzc25lj8c+6JybkL39qL8yi4jIG7zSMIGqzPq+6dYu0q2ddP5pOe2/WkR81TZcMn1Knje1t534yq2EJtSq4O0fx7usqAI4A3jWzDYC5wCP+zmZVez1RlxHnOjcBl1PKCIn5brrrsPMMDNCoRDDhg1j3rx5fPOb36S9vd3veNJHuihKRER6LVhTSfl7zqLkwskQ8Oh6YS1tv5iPSyRz+jzOObpeXg+hIJGZY3PathzVMS9Jcs7tc84Nc87VO+fqgZeAK5xzC/0Im26PEV/aSLB+GMHaQcd/gIjIUZx//vls27aNTZs28ec//5lrrrmG7373u8yaNYvm5ma/4/kmHo/7HaHPVPSKiMhJMc8INdRQdsVMSi85k/DpI7FQ5qqZ+OrtpFv7PulVsnE3qa17icwYo1HefuKcSwIHLklaAfz8wCVJ2cuQ8kp85TbAEZ0zzu8oIlLgwuEww4cPZ+TIkZx55pl84hOf4MUXX2THjh187nMHr972ne98h9NOO41oNMrEiRO56667SCYzX/x+4QtfYPLkyYe1/4lPfIJ58+Yd9fmfeeYZLrroIoYMGcKgQYO48MILmT9//kHHtLW1cfPNNzN69GgikQj19fV89atf7d7f0tLChz/8YWpra4lGo0yePJkHH3wQgGeffRYzo7Gx8aA2g8EgP/7xjwHYuHEjZsbDDz/M5ZdfTllZGbfddhvOOT72sY8xfvx4SkpKaGho4NZbbyUWO3gehT/84Q+cf/75lJaWdr+GdevW8eyzzxIIBNiyZctBxz/00EMMGjTolI+m98s1vSIiUrzMjODIwQRHZmbMTXcl6HppLaQdwbHDiJwxikB1xUm1nd7TgTe4lPDpI45/sOSMc+5J4MlDtt1+lGMv6o9MRxOZOZbQ2KF4FVE/Y4jIUbT/bslh20L1wwifPhKXTNHxzLLD90+oJTyxlnRXgs4/rzhsf3jyCEIN1aTbYnT+ddVh+8sum5ab8EBdXR3XXHMNDz30ED/84Q/xPI8777yTH/3oR9x9993MmDGDFStW8PGPf5yuri6+/OUvc+211/LVr36Vl19+mbPPPhuAWCzGo48+yte//vWjPldbWxv/+I//yPTp00kmk3z729/m0ksvZc2aNQwdOhTnHO94xzvYvHkz3/nOd5g2bRqNjY2sWpX5HXR2dnLhhRdSUlLCww8/TENDA2vXrmX37t5Ph3TLLbfwjW98g3vvvRfInHlVU1PDI488Qm1tLUuWLOGGG24gFArxxS9+EcgUvJdccgmf+tSn+O53v0skEuH5558nkUhw0UUXMXHiRB588EHuuOOO7ue5//77+Yd/+AfKysp6nbE3VPSKiEhOedEQ5e89i/jyrcRXbSO5cSeB2kqib5pAYHDvOrXItNGEp9Z1L6MkcoBzDhIpLBwkMOzkvlQRETkRU6dOZf/+/ezcuZPy8nK++c1v8stf/pJLL70UgHHjxvGVr3yFm266iS9/+ctMmjSJs88+m4ceeqi76P3Nb35DZ2cn73vf+476PO9617sOun/ffffxv//7vzz11FNcc801/OlPf+K5555jwYIFzJmTmUahoaGBCy64AIBHHnmEDRs2sHbtWkaNGtW9/2TccMMNXHPNNQdtu+uuu7p/rq+vZ926dXzve9/rLnq/+MUvctlll3H33Xd3H3faaad1/3z99ddzzz33cNttt+F5HitXruRvf/sb//mf/3lSGXtDRa+IiOScVxohOmcckemjia9uJrFyGxbJnJ6cbuvComEsePRCNt0ZJ72/k2DtIK3JK0eU3LSTrhfWUnrZtO4J1kQk/xxr1NWCgWPu96KhY+8vj+R0VPdonMtMPG9mLFu2jM7OTt7znvccNHFeKpWiq6uLHTt2UF1dzbXXXsttt93G3XffTSgU4qGHHuKKK65g8OCjryO+YcMGbr/9dl588UVaWlpIp9N0dHSwadMmABYtWkRVVVV3wXuoRYsWMWXKlO6Cty/mzp172Lb777+fBx54gI0bN9Le3k4ymSSdfmMSy0WLFh1zJPvaa6/lC1/4Ak8//TSXXXYZDzzwALNnz2bmzJl9zns8+iQhIiKnjIWCRKbWUfbu2XglmaVkOp9bRdsv5hN7bXP3EkiHir2yiY6nlpLuKPzJMyT3XDJN14LMEkXeoFK/44hIkVu2bBmDBg1i6NCh3UXeY489xuLFi7tvS5cuZc2aNQwZMgSAq666itbWVp544gl27NjBU089xbXXXnvM5zlw6vK9997LSy+9xOLFi6mpqcnZRFJe9qypA0U8ZIr1noXrAYeebvzYY49x44038v73v58nn3ySV199ldtvv51E4sj9+JEMHTqU9773vdx///3E43Eeeughrr/++pN8Nb2jkV4RETnlDnwb7pwjMnMMsdebiL2yidiSLYQnDic8ta77mszUrjYSq7cTnjIST2uuyhHEVzTh2rooueQMzNMSRSJy6jQ1NfHwww/z7ne/G8/zmDp1KtFolPXr13P55Zcf9XFVVVW8853v5L//+7/ZvHkzQ4YM4ZJLLjnq8bt27WL58uU8+eST3cc1NjbS0tLSfczs2bPZs2cPCxcuPOJo7+zZs3nwwQdpbGw84mhvTU0NAFu3bmX06MzKdIsXLz6oCD6av/zlL8ycOZPPfOYz3ds2btx42PP//ve/56abbjpqOzfccANvfvOb+cEPfkBnZydXX331cZ87FzTSKyIi/SYz6VUVZRefQdmVswiNHUZ85TaSW3YBB5YoWodFgkRmjPE5reSjdGec2GtbCI4eQnBkld9xRKSIxONxtm/fztatW1m6dCnf//73Offcc6mpqeFrX/saAOXl5dx6663ceuut3HvvvaxatYply5bxs5/9jFtuueWg9j70oQ/x29/+lv/6r//immuuIRAIHPW5q6qqqK6u5v7772f16tW8+OKLXH311ZSUlHQf85a3vIXzzz+f97///fz6179mw4YNPP/88zzwwAMAXH311YwdO5YrrriCP/zhD2zYsIE//vGPPProowBMmDCBsWPHcuedd3ZfT/vpT3/6hNY3nzx5MkuXLuXXv/4169at45577uGXv/zlQcfcdttt/O53v+Pmm29myZIlrFq1ih//+MfdE20BzJs3j8mTJ/PZz36Wq666ioqK/pmTQUWviIj4IjCkjJILJlP+92cRmjgcgNjizaSa9xOZVd99DbBIT8lNOyGZJnKWligSkdz661//yogRIxgzZgwXXXQRDz/8MJ/85Cd55ZVXqK2t7T7utttu41vf+hb3338/06dPZ968eXz729+mvr7+oPYuu+wyBg0axIoVK/jQhz50zOf2PI/HHnuMdevWMW3aNK677jpuvvlmRox4Y/UCM+OJJ57g8ssv5+Mf/ziTJ0/mAx/4ADt37gSgtLSU5557jjPOOIOrrrqK008/nRtvvJHOzswSgsFgkEcffZSWlhZmzpzJjTfeyF133dV92vOx3HDDDXzwgx/kwx/+MDNnzuTll1/mzjvvPOiYiy++mCeffLJ71uq5c+fyk5/8hFDo4P78Yx/7GPF4vN9ObQawExnOLkRz5sxxCxcu9DuGiIj0QnxtM6kdrUTPHp9Xp62a2SLn3JFnDpETlqu+Ob2/E6+y5PgHiki/WrFiBaeffrrfMSTP/cu//AvPPPMMr7766nGPPdZ7qjd9s67pFRGRvBGeUAsTao9/oAxoKnhFRArPvn37WL16Nffdd1+/LFPUk05vFhERERERkVPqyiuv5IILLuBd73oXH/jAB/r1uTXSKyIiIiIiIqfUs88+69tza6RXREREREREipaKXhERERERyYlinSRX+l8u30sqekVEREREpM9CoVD38jgifdXZ2XnYckcnS0WviIiIiIj0WU1NDU1NTXR0dGjEV06ac46Ojg6ampqoqanJSZuayEpERERERPqssrISgK1bt5JIJHxOI4UsFApRW1vb/Z7qKxW9IiIiIiKSE5WVlTkrVERyRac3i4iIiIiISNFS0SsiIiIiIiJFS0WviIiIiIiIFC0VvSIiIiIiIlK0VPSKiIiIiIhI0bJiXUPLzHYAm3LQ1DBgZw7a8UOhZi/U3KDsfinU7IWaGwZe9rHOuepTEWYgyWHf7IdCfs+fqIHwGkGvs5gMhNcIep1Hc8J9c9EWvbliZgudc3P8znEyCjV7oeYGZfdLoWYv1Nyg7DLwDIT3zUB4jaDXWUwGwmsEvc5c0OnNIiIiIiIiUrRU9IqIiIiIiEjRUtF7fPf5HaAPCjV7oeYGZfdLoWYv1Nyg7DLwDIT3zUB4jaDXWUwGwmsEvc4+0zW9IiIiIiIiUrQ00isiIiIiIiJFS0WviIiIiIiIFC0VvcdgZpea2SozW2tmn/M7z7GY2YNm1mJmr/fYNsTMnjGzNdl/q/zMeCRmNtrM/mxmy81smZn9U3Z7IWSPmtl8M3stm/2L2e3jzOzl7PvmUTML+531SMwsYGavmtlvs/cLJfdGM1tqZovNbGF2W96/XwDMbLCZ/cLMVprZCjM7N9+zm9nk7O/6wG2/md2c77kPMLNPZ///fN3Mfpr9/7Yg3uvij0Lul3qj0Puw3ijU/q43Crlv7I1C7Ed7q9D73RPV3/2zit6jMLMAcC9wGTAFuNrMpvib6ph+DFx6yLbPAX90zk0E/pi9n2+SwP9zzk0BzgFuzP6eCyF7DHiLc246MAO41MzOAb4BfNs5NwHYA3zEx4zH8k/Aih73CyU3wJudczN6rOVWCO8XgHuAp5xzpwHTyfz+8zq7c25V9nc9A5gNdAC/Is9zA5hZHXATMMc5dwYQAK6isN7r0v8KuV/qjULvw3qjkPu73ijUvrE3Cq4f7a1C7ndPlC/9s3NOtyPcgHOBp3vc/zzweb9zHSdzPfB6j/urgBHZn0cAq/zOeAKv4dfA2wstO1AKvAKcDewEgkd6H+XLDRhF5g/mW4DfAlYIubPZNgLDDtmW9+8XYBCwgewEgoWUvUfWi4HnCyU3UAdsAYYAwex7/ZJCea/rlh+3Qu2XevkaC6oP6+VrK9j+rpevsyD7xl6+xoLvR0/iNRdUv9uL19Xv/bNGeo/uwH+MAxqz2wpJrXNuW/bn7UCtn2GOx8zqgZnAyxRI9uwpU4uBFuAZYB2w1zmXzB6Sr++bu4F/AdLZ+0MpjNwADvi9mS0ys+uz2wrh/TIO2AH8KHua3QNmVkZhZD/gKuCn2Z/zPrdzrgn4d2AzsA3YByyicN7r4rNC7Jd6o4D7sN4o5P6uNwq1b+yNYuhHe6ug+t0T5Uf/rKJ3gHCZr0zydn0qMysH/he42Tm3v+e+fM7unEu5zOkno4C5wGk+RzouM3sH0OKcW+R3lpM0zzk3i8ylBzea2QU9d+bx+yUIzAK+75ybCbRzyKlJeZyd7HU1VwCPHbovX3Nnr3e6kswHpZFAGYdfBiJyRIXaL/VGIfZhvVEE/V1vFGrf2BsF3Y/2ViH2uyfKj/5ZRe/RNQGje9wfld1WSJrNbARA9t8Wn/MckZmFyHyweNg598vs5oLIfoBzbi/wZzKnYgw2s2B2Vz6+b84DrjCzjcDPyJzydQ/5nxvo/nYQ51wLmWtc5lIY75dGoNE593L2/i/IdN6FkB0yH6Recc41Z+8XQu63ARucczuccwngl2Te/wXxXhf/FEO/1BsF1of1RkH3d71RwH1jbxR6P9pbhdjvnqh+759V9B7dAmBidhaxMJnTCx73OVNvPQ5cm/35WjLXJeUVMzPgh8AK59y3euwqhOzVZjY4+3MJmWu+VpD54PDe7GF5l90593nn3CjnXD2Z9/WfnHPXkOe5AcyszMwqDvxM5lqX1ymA94tzbjuwxcwmZze9FVhOAWTPupo3TrGCwsi9GTjHzEqzf2sO/M7z/r0u/inkfqk3CrUP641C7u96o5D7xt4ogn60twqx3z1R/d4/W/ZCYTkCM7uczLUgAeBB59xdPkc6KjP7KXARMAxoBu4A/g/4OTAG2AS8zzm326+MR2Jm84C/Akt543qbW8lcP5Xv2acBPyHz/vCAnzvnvmRmDWS+UR4CvAp8wDkX8y/p0ZnZRcBnnXPvKITc2Yy/yt4NAo845+4ys6Hk+fsFwMxmAA8AYWA98GGy7x3yOHv2Q9RmoME5ty+7rVB+518E3k9mRt5XgY+SuUYor9/r4p9C7pd6oxj6sN4otP6uNwq9b+yNQu1He6uQ+90T1d/9s4peERERERERKVo6vVlERERERESKlopeERERERERKVoqekVERERERKRoqegVERERERGRoqWiV0RERERERIqWil4REREREREpWip6RQYAM7vTzP7H7xwiIiKSob5ZpP+o6BUREREREZGipaJXpMiY2S1m1mRmrWa2ysz+DrgVeL+ZtZnZa9njBpnZD81sW/b4r5hZILvvOjN73sy+a2b7zGylmb3Vz9clIiJSqNQ3i/gr6HcAEckdM5sMfBI4yzm31czqgQDwVWCCc+4DPQ7/MdACTADKgN8CW4AfZPefDfwCGAa8G/ilmY1zzu0+9a9ERESkOKhvFvGfRnpFiksKiABTzCzknNvonFt36EFmVgtcDtzsnGt3zrUA3wau6nFYC3C3cy7hnHsUWAX83al/CSIiIkVFfbOIzzTSK1JEnHNrzexm4E5gqpk9DXzmCIeOBULANjM7sM0j823yAU3OOdfj/iZgZM5Di4iIFDH1zSL+00ivSJFxzj3inJtHpvN0wDey//a0BYgBw5xzg7O3Sufc1B7H1FmPXhcYA2w9ldlFRESKkfpmEX+p6BUpImY22czeYmYRoAvoBNJAM1BvZh6Ac24b8HvgP8ys0sw8MxtvZhf2aK4GuMnMQmb298DpwJP9+oJEREQKnPpmEf+p6BUpLhHg68BOYDuZzvHzwGPZ/bvM7JXszx8CwsByYA+ZiTFG9GjrZWBitq27gPc653ad6hcgIiJSZNQ3i/jMDr4sQEQksywC8NHsqVgiIiLiM/XNIidPI70iIiIiIiJStFT0ioiIiIiISNHS6c0iIiIiIiJStDTSKyIiIiIiIkVLRa+IiIiIiIgULRW9IiIiIiIiUrRU9IqIiIiIiEjRUtErIiIiIiIiRev/A7PhuKF6dNBbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from nndl import plot_training_loss_acc\n",
    "plot_training_loss_acc(runner, 'cnn-loss1.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.3.4 模型评价\n",
    "\n",
    "使用测试数据对在训练过程中保存的最佳模型进行评价，观察模型在测试集上的准确率以及损失变化情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Test] accuracy/loss: 0.8700/0.4423\n"
     ]
    }
   ],
   "source": [
    "# 加载最优模型\n",
    "runner.load_model('best_model.pdparams')\n",
    "# 模型评价\n",
    "score, loss = runner.evaluate(test_loader)\n",
    "print(\"[Test] accuracy/loss: {:.4f}/{:.4f}\".format(score, loss))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.3.5 模型预测\n",
    "\n",
    "同样地，我们也可以使用保存好的模型，对测试集中的某一个数据进行模型预测，观察模型效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The true category is 2 and the predicted category is 2\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAJIAAACPCAYAAAARM4LLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAABZtJREFUeJzt3L+LHHUYx/H3x2gqG+OFEJLgpghCOiGIop0GzjSxkqSQFAEbBQWbRP8BKzubgCEpJCIoJEUgaBBEELk7CZofXBKFkAvRXLDQ0sBjsWPYW9zs3M5zM7M7nxcstzO7x3yLz37nme/sPooIzKp6rOkB2GxwkCyFg2QpHCRL4SBZCgfJUjhIlsJBshSVgiRpXtKypJuSjmUNyqaPJl3ZlrQJuA7sB1aABeBwRFwd9T9zc3PR6/UmOp41Y2lp6X5EbB33vscrHON54GZE/AYg6XPgIDAySL1ej8XFxQqHtLpJulXmfVVObTuA2wPbK8W+4YG8JWlR0uLq6mqFw1mbbXixHREnImJfROzbunXsDGlTqkqQ7gC7BrZ3Fvusg6oEaQHYI2m3pM3AIeBczrBs2kxcbEfEA0nvABeATcDJiLiSNjKbKlWu2oiI88D5pLHYFPPKtqVwkCyFg2QpHCRLUanYniWSajvWLP5yxzOSpXCQLIWDZCk6WyPVWRONO/Ys1EyekSyFg2QpOnNqa/JU1gWekSyFg2QpHCRLMbM10kbXRIOX7K6/PCNZEgfJUjhIlmJma6SqZuG2RZ08I1kKB8lSOEiWYmZrpOEaZ9xaj2uiajwjWQoHyVI4SJZiZmukYdk1kO+vreUZyVKMDZKkk5LuSbo8sG+LpK8l3Sj+PrWxw7S2KzMjnQLmh/YdAy5GxB7gYrFtHTY2SBHxHfDn0O6DwOni+Wng9eRxtY6kNY8qImLNYxZMWiNti4i7xfPfgW1J47EpVbnYjv5HauTHyu2Ru2HSIP0haTtA8ffeqDe6PXI3TBqkc8CR4vkR4GzOcGbTLNZEw8pc/p8BfgCelbQi6SjwEbBf0g3g1WLbOmzsynZEHB7x0ivJY7Ep5pVtS9GZe23r5Xtp6+MZyVI4SJbCQbIUrpEKmTXRrK4VPYpnJEvhIFkKn9oSdPFUNswzkqVwkCyFg2QpOlsj+RZILs9IlsJBshQOkqXoTI2UXRN57Wgtz0iWwkGyFA6SpXCQLIWDZCkcJEvhIFmKzqwjVeV1o0fzjGQpHCRL4SBZCtdIJW3kz5Xa9N2oSWtBz0iWokx/pF2SvpV0VdIVSe8W+90i2R4qMyM9AN6PiL3AC8DbkvbiFsk2oEx75LsR8VPx/G/gGrCDDrZIzpLZarkt1lUjSeoBzwE/4hbJNqB0kCQ9CXwJvBcRfw2+9qgWyW6P3A2lgiTpCfoh+iwivip2l2qR7PbI3VDmqk3Ap8C1iPh44CW3SLaHyixIvgS8Cfwi6VKx7wP6LZG/KNol3wLe2Jgh2jQo0x75e2DUpYVbJBvglW1L0pl7bW2+v7VebfxulGckS+EgWQoHyVJ0pkYa1sY6Y5p5RrIUDpKlcJAshYNkKRwkS+EgWQoHyVI4SJbCQbIUDpKlcJAshYNkKRwkS+EgWQoHyVI4SJbCQbIUDpKlUJ1fOZW0Sv9XuXPA/doOvD5tHVtT43omIsY2bag1SA8PKi1GxL7aD1xCW8fW1nH9x6c2S+EgWYqmgnSioeOW0daxtXVcQEM1ks0en9osRa1BkjQvaVnSTUmNtlOWdFLSPUmXB/a1onf4NPY2ry1IkjYBnwCvAXuBw0W/7qacAuaH9rWld/j09TaPiFoewIvAhYHt48Dxuo4/Ykw94PLA9jKwvXi+HVhucnwD4zoL7G/r+CKi1lPbDuD2wPZKsa9NWtc7fFp6m7vYHiH6H/tGL2kn7W3ehDqDdAfYNbC9s9jXJqV6h9ehSm/zJtQZpAVgj6TdkjYDh+j36m6TVvQOn8re5jUXjQeA68CvwIcNF7BngLvAP/TrtaPA0/Svhm4A3wBbGhrby/RPWz8Dl4rHgbaM7/8eXtm2FC62LYWDZCkcJEvhIFkKB8lSOEiWwkGyFA6SpfgXPjuVh8Q+xYUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 144x144 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 获取测试集中第一条数据\n",
    "X, label = next(test_loader())\n",
    "logits = runner.predict(X)\n",
    "# 多分类，使用softmax计算预测概率\n",
    "pred = F.softmax(logits)\n",
    "# 获取概率最大的类别\n",
    "pred_class = paddle.argmax(pred[1]).numpy()\n",
    "label = label[1][0].numpy()\n",
    "# 输出真实类别与预测类别\n",
    "print(\"The true category is {} and the predicted category is {}\".format(label[0], pred_class[0]))\n",
    "# 可视化图片\n",
    "plt.figure(figsize=(2, 2))\n",
    "image, label = test_set[0][1], test_set[1][1]\n",
    "image= np.array(image).astype('float32')\n",
    "image = np.reshape(image, [28,28])\n",
    "image = Image.fromarray(image.astype('uint8'), mode='L')\n",
    "plt.imshow(image)\n",
    "plt.savefig('cnn-number2.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 5.4 基于残差网络的手写体数字识别实验"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**残差网络**（Residual Network，ResNet）是在神经网络模型中给非线性层增加直连边的方式来缓解梯度消失问题，从而使训练深度神经网络变得更加容易。\n",
    "\n",
    "在残差网络中，最基本的单位为**残差单元**。\n",
    "\n",
    "假设$f(\\mathbf x;\\theta)$为一个或多个神经层，残差单元在$f()$的输入和输出之间加上一个**直连边**。\n",
    "\n",
    "不同于传统网络结构中让网络$f(x;\\theta)$去逼近一个目标函数$h(x)$，在残差网络中，将目标函数$h(x)$拆为了两个部分：恒等函数$x$和残差函数$h(x)-x$\n",
    "\n",
    "$$\n",
    "\\mathrm{ResBlock}_f(\\mathbf x) = f(\\mathbf x;\\theta) + \\mathbf x,（5.22）\n",
    "$$\n",
    "\n",
    "其中$\\theta$为可学习的参数。\n",
    "\n",
    "一个典型的残差单元如**图5.14**所示，由多个级联的卷积层和一个跨层的直连边组成。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/cf14626bb6c54eb9840a7253975fdc995f165146c6324bdb8f9b943b96f57394\" width = \"200\"></center>\n",
    "<center><br>图5.14：残差单元结构</br></center>\n",
    "\n",
    "\n",
    "一个残差网络通常有很多个残差单元堆叠而成。下面我们来构建一个在计算机视觉中非常典型的残差网络：ResNet18，并重复上一节中的手写体数字识别任务。\n",
    "\n",
    "### 5.4.1 模型构建\n",
    "\n",
    "在本节中，我们先构建ResNet18的残差单元，然后在组建完整的网络。\n",
    "\n",
    "#### 5.4.1.1 残差单元\n",
    "\n",
    "这里，我们实现一个算子`ResBlock`来构建残差单元，其中定义了`use_residual`参数，用于在后续实验中控制是否使用残差连接。\n",
    "\n",
    "\n",
    "*******\n",
    "\n",
    "残差单元包裹的非线性层的输入和输出形状大小应该一致。如果一个卷积层的输入特征图和输出特征图的通道数不一致，则其输出与输入特征图无法直接相加。为了解决上述问题，我们可以使用$1 \\times 1$大小的卷积将输入特征图的通道数映射为与级联卷积输出特征图的一致通道数。\n",
    "\n",
    "$1 \\times 1$卷积：与标准卷积完全一样，唯一的特殊点在于卷积核的尺寸是$1 \\times 1$，也就是不去考虑输入数据局部信息之间的关系，而把关注点放在不同通道间。通过使用$1 \\times 1$卷积，可以起到如下作用：\n",
    "\n",
    "* 实现信息的跨通道交互与整合。考虑到卷积运算的输入输出都是3个维度（宽、高、多通道），所以$1 \\times 1$卷积实际上就是对每个像素点，在不同的通道上进行线性组合，从而整合不同通道的信息；\n",
    "* 对卷积核通道数进行降维和升维，减少参数量。经过$1 \\times 1$卷积后的输出保留了输入数据的原有平面结构，通过调控通道数，从而完成升维或降维的作用；\n",
    "* 利用$1 \\times 1$卷积后的非线性激活函数，在保持特征图尺寸不变的前提下，大幅增加非线性。\n",
    "\n",
    "*******"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class ResBlock(nn.Layer):\n",
    "    def __init__(self, in_channels, out_channels, stride=1, use_residual=True):\n",
    "        \"\"\"\n",
    "        残差单元\n",
    "        输入：\n",
    "            - in_channels：输入通道数\n",
    "            - out_channels：输出通道数\n",
    "            - stride：残差单元的步长，通过调整残差单元中第一个卷积层的步长来控制\n",
    "            - use_residual：用于控制是否使用残差连接\n",
    "        \"\"\"\n",
    "        super(ResBlock, self).__init__()\n",
    "        self.stride = stride\n",
    "        self.use_residual = use_residual\n",
    "        # 第一个卷积层，卷积核大小为3×3，可以设置不同输出通道数以及步长\n",
    "        self.conv1 = nn.Conv2D(in_channels, out_channels, 3, padding=1, stride=self.stride, bias_attr=False)\n",
    "        # 第二个卷积层，卷积核大小为3×3，不改变输入特征图的形状，步长为1\n",
    "        self.conv2 = nn.Conv2D(out_channels, out_channels, 3, padding=1, bias_attr=False)\n",
    "        \n",
    "        # 如果conv2的输出和此残差块的输入数据形状不一致，则use_1x1conv = True\n",
    "        # 当use_1x1conv = True，添加1个1x1的卷积作用在输入数据上，使其形状变成跟conv2一致\n",
    "        if in_channels != out_channels or stride != 1:\n",
    "            self.use_1x1conv = True\n",
    "        else:\n",
    "            self.use_1x1conv = False\n",
    "        # 当残差单元包裹的非线性层输入和输出通道数不一致时，需要用1×1卷积调整通道数后再进行相加运算\n",
    "        if self.use_1x1conv:\n",
    "            self.shortcut = nn.Conv2D(in_channels, out_channels, 1, stride=self.stride, bias_attr=False)\n",
    "\n",
    "        # 每个卷积层后会接一个批量规范化层，批量规范化的内容在7.5.1中会进行详细介绍\n",
    "        self.bn1 = nn.BatchNorm2D(out_channels)\n",
    "        self.bn2 = nn.BatchNorm2D(out_channels)\n",
    "        if self.use_1x1conv:\n",
    "            self.bn3 = nn.BatchNorm2D(out_channels)\n",
    "\n",
    "    def forward(self, inputs):\n",
    "        y = F.relu(self.bn1(self.conv1(inputs)))\n",
    "        y = self.bn2(self.conv2(y))\n",
    "        if self.use_residual:\n",
    "            if self.use_1x1conv:  # 如果为真，对inputs进行1×1卷积，将形状调整成跟conv2的输出y一致\n",
    "                shortcut = self.shortcut(inputs)\n",
    "                shortcut = self.bn3(shortcut)\n",
    "            else: # 否则直接将inputs和conv2的输出y相加\n",
    "                shortcut = inputs\n",
    "            y = paddle.add(shortcut, y)\n",
    "        out = F.relu(y)\n",
    "        return out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 5.4.1.2 残差网络的整体结构\n",
    "\n",
    "残差网络就是将很多个残差单元串联起来构成的一个非常深的网络。ResNet18 的网络结构如**图5.16**所示。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/16cb9fe0d0704940a829be1722fd6273b85209f198d143efb221f760d5f87f64\" width = \"700\"></center>\n",
    "<center><br>图5.16：残差网络</br></center>\n",
    "\n",
    "其中为了便于理解，可以将ResNet18网络划分为6个模块：\n",
    "\n",
    "* 第一模块：包含了一个步长为2，大小为$7 \\times 7$的卷积层，卷积层的输出通道数为64，卷积层的输出经过批量归一化、ReLU激活函数的处理后，接了一个步长为2的$3 \\times 3$的最大汇聚层；\n",
    "* 第二模块：包含了两个残差单元，经过运算后，输出通道数为64，特征图的尺寸保持不变；\n",
    "* 第三模块：包含了两个残差单元，经过运算后，输出通道数为128，特征图的尺寸缩小一半；\n",
    "* 第四模块：包含了两个残差单元，经过运算后，输出通道数为256，特征图的尺寸缩小一半；\n",
    "* 第五模块：包含了两个残差单元，经过运算后，输出通道数为512，特征图的尺寸缩小一半；\n",
    "* 第六模块：包含了一个全局平均汇聚层，将特征图变为$1 \\times 1$的大小，最终经过全连接层计算出最后的输出。\n",
    "\n",
    "ResNet18模型的代码实现如下："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "定义模块一。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def make_first_module(in_channels):\n",
    "    # 模块一：7*7卷积、批量规范化、汇聚\n",
    "    m1 = nn.Sequential(nn.Conv2D(in_channels, 64, 7, stride=2, padding=3),\n",
    "                    nn.BatchNorm2D(64), nn.ReLU(),\n",
    "                    nn.MaxPool2D(kernel_size=3, stride=2, padding=1))\n",
    "    return m1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "定义模块二到模块五。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def resnet_module(input_channels, out_channels, num_res_blocks, stride=1, use_residual=True):\n",
    "    blk = []\n",
    "    # 根据num_res_blocks，循环生成残差单元\n",
    "    for i in range(num_res_blocks):\n",
    "        if i == 0: # 创建模块中的第一个残差单元\n",
    "            blk.append(ResBlock(input_channels, out_channels,\n",
    "                                stride=stride, use_residual=use_residual))\n",
    "        else:      # 创建模块中的其他残差单元\n",
    "            blk.append(ResBlock(out_channels, out_channels, use_residual=use_residual))\n",
    "    return blk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "封装模块二到模块五。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def make_modules(use_residual):\n",
    "    # 模块二：包含两个残差单元，输入通道数为64，输出通道数为64，步长为1，特征图大小保持不变\n",
    "    m2 = nn.Sequential(*resnet_module(64, 64, 2, stride=1, use_residual=use_residual))\n",
    "    # 模块三：包含两个残差单元，输入通道数为64，输出通道数为128，步长为2，特征图大小缩小一半。\n",
    "    m3 = nn.Sequential(*resnet_module(64, 128, 2, stride=2, use_residual=use_residual))\n",
    "    # 模块四：包含两个残差单元，输入通道数为128，输出通道数为256，步长为2，特征图大小缩小一半。\n",
    "    m4 = nn.Sequential(*resnet_module(128, 256, 2, stride=2, use_residual=use_residual))\n",
    "    # 模块五：包含两个残差单元，输入通道数为256，输出通道数为512，步长为2，特征图大小缩小一半。\n",
    "    m5 = nn.Sequential(*resnet_module(256, 512, 2, stride=2, use_residual=use_residual))\n",
    "    return m2, m3, m4, m5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "定义完整网络。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 定义完整网络\n",
    "class Model_ResNet18(nn.Layer):\n",
    "    def __init__(self, in_channels=3, num_classes=10, use_residual=True):\n",
    "        super(Model_ResNet18,self).__init__()\n",
    "        m1 = make_first_module(in_channels)\n",
    "        m2, m3, m4, m5 = make_modules(use_residual)\n",
    "        # 封装模块一到模块6\n",
    "        self.net = nn.Sequential(m1, m2, m3, m4, m5,\n",
    "                        # 模块六：汇聚层、全连接层\n",
    "                        nn.AdaptiveAvgPool2D(1), nn.Flatten(), nn.Linear(512, num_classes) )\n",
    "\n",
    "    def forward(self, x):\n",
    "        return self.net(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "这里同样可以使用`paddle.summary`统计模型的参数量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-------------------------------------------------------------------------------\n",
      "   Layer (type)         Input Shape          Output Shape         Param #    \n",
      "===============================================================================\n",
      "     Conv2D-22        [[1, 1, 32, 32]]     [1, 64, 16, 16]         3,200     \n",
      "   BatchNorm2D-1     [[1, 64, 16, 16]]     [1, 64, 16, 16]          256      \n",
      "      ReLU-1         [[1, 64, 16, 16]]     [1, 64, 16, 16]           0       \n",
      "    MaxPool2D-3      [[1, 64, 16, 16]]      [1, 64, 8, 8]            0       \n",
      "     Conv2D-23        [[1, 64, 8, 8]]       [1, 64, 8, 8]         36,864     \n",
      "   BatchNorm2D-2      [[1, 64, 8, 8]]       [1, 64, 8, 8]           256      \n",
      "     Conv2D-24        [[1, 64, 8, 8]]       [1, 64, 8, 8]         36,864     \n",
      "   BatchNorm2D-3      [[1, 64, 8, 8]]       [1, 64, 8, 8]           256      \n",
      "    ResBlock-1        [[1, 64, 8, 8]]       [1, 64, 8, 8]            0       \n",
      "     Conv2D-25        [[1, 64, 8, 8]]       [1, 64, 8, 8]         36,864     \n",
      "   BatchNorm2D-4      [[1, 64, 8, 8]]       [1, 64, 8, 8]           256      \n",
      "     Conv2D-26        [[1, 64, 8, 8]]       [1, 64, 8, 8]         36,864     \n",
      "   BatchNorm2D-5      [[1, 64, 8, 8]]       [1, 64, 8, 8]           256      \n",
      "    ResBlock-2        [[1, 64, 8, 8]]       [1, 64, 8, 8]            0       \n",
      "     Conv2D-27        [[1, 64, 8, 8]]       [1, 128, 4, 4]        73,728     \n",
      "   BatchNorm2D-6      [[1, 128, 4, 4]]      [1, 128, 4, 4]          512      \n",
      "     Conv2D-28        [[1, 128, 4, 4]]      [1, 128, 4, 4]        147,456    \n",
      "   BatchNorm2D-7      [[1, 128, 4, 4]]      [1, 128, 4, 4]          512      \n",
      "     Conv2D-29        [[1, 64, 8, 8]]       [1, 128, 4, 4]         8,192     \n",
      "   BatchNorm2D-8      [[1, 128, 4, 4]]      [1, 128, 4, 4]          512      \n",
      "    ResBlock-3        [[1, 64, 8, 8]]       [1, 128, 4, 4]           0       \n",
      "     Conv2D-30        [[1, 128, 4, 4]]      [1, 128, 4, 4]        147,456    \n",
      "   BatchNorm2D-9      [[1, 128, 4, 4]]      [1, 128, 4, 4]          512      \n",
      "     Conv2D-31        [[1, 128, 4, 4]]      [1, 128, 4, 4]        147,456    \n",
      "  BatchNorm2D-10      [[1, 128, 4, 4]]      [1, 128, 4, 4]          512      \n",
      "    ResBlock-4        [[1, 128, 4, 4]]      [1, 128, 4, 4]           0       \n",
      "     Conv2D-32        [[1, 128, 4, 4]]      [1, 256, 2, 2]        294,912    \n",
      "  BatchNorm2D-11      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     \n",
      "     Conv2D-33        [[1, 256, 2, 2]]      [1, 256, 2, 2]        589,824    \n",
      "  BatchNorm2D-12      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     \n",
      "     Conv2D-34        [[1, 128, 4, 4]]      [1, 256, 2, 2]        32,768     \n",
      "  BatchNorm2D-13      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     \n",
      "    ResBlock-5        [[1, 128, 4, 4]]      [1, 256, 2, 2]           0       \n",
      "     Conv2D-35        [[1, 256, 2, 2]]      [1, 256, 2, 2]        589,824    \n",
      "  BatchNorm2D-14      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     \n",
      "     Conv2D-36        [[1, 256, 2, 2]]      [1, 256, 2, 2]        589,824    \n",
      "  BatchNorm2D-15      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     \n",
      "    ResBlock-6        [[1, 256, 2, 2]]      [1, 256, 2, 2]           0       \n",
      "     Conv2D-37        [[1, 256, 2, 2]]      [1, 512, 1, 1]       1,179,648   \n",
      "  BatchNorm2D-16      [[1, 512, 1, 1]]      [1, 512, 1, 1]         2,048     \n",
      "     Conv2D-38        [[1, 512, 1, 1]]      [1, 512, 1, 1]       2,359,296   \n",
      "  BatchNorm2D-17      [[1, 512, 1, 1]]      [1, 512, 1, 1]         2,048     \n",
      "     Conv2D-39        [[1, 256, 2, 2]]      [1, 512, 1, 1]        131,072    \n",
      "  BatchNorm2D-18      [[1, 512, 1, 1]]      [1, 512, 1, 1]         2,048     \n",
      "    ResBlock-7        [[1, 256, 2, 2]]      [1, 512, 1, 1]           0       \n",
      "     Conv2D-40        [[1, 512, 1, 1]]      [1, 512, 1, 1]       2,359,296   \n",
      "  BatchNorm2D-19      [[1, 512, 1, 1]]      [1, 512, 1, 1]         2,048     \n",
      "     Conv2D-41        [[1, 512, 1, 1]]      [1, 512, 1, 1]       2,359,296   \n",
      "  BatchNorm2D-20      [[1, 512, 1, 1]]      [1, 512, 1, 1]         2,048     \n",
      "    ResBlock-8        [[1, 512, 1, 1]]      [1, 512, 1, 1]           0       \n",
      "AdaptiveAvgPool2D-1   [[1, 512, 1, 1]]      [1, 512, 1, 1]           0       \n",
      "     Flatten-1        [[1, 512, 1, 1]]         [1, 512]              0       \n",
      "     Linear-11           [[1, 512]]            [1, 10]             5,130     \n",
      "===============================================================================\n",
      "Total params: 11,185,034\n",
      "Trainable params: 11,165,834\n",
      "Non-trainable params: 19,200\n",
      "-------------------------------------------------------------------------------\n",
      "Input size (MB): 0.00\n",
      "Forward/backward pass size (MB): 1.05\n",
      "Params size (MB): 42.67\n",
      "Estimated Total Size (MB): 43.73\n",
      "-------------------------------------------------------------------------------\n",
      "\n",
      "{'total_params': 11185034, 'trainable_params': 11165834}\n"
     ]
    }
   ],
   "source": [
    "model = Model_ResNet18(in_channels=1, num_classes=10, use_residual=True)\n",
    "params_info = paddle.summary(model, (1, 1, 32, 32))\n",
    "print(params_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "使用`paddle.flops`统计模型的计算量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'paddle.nn.layer.conv.Conv2D'>'s flops has been counted\n",
      "<class 'paddle.nn.layer.norm.BatchNorm2D'>'s flops has been counted\n",
      "<class 'paddle.nn.layer.activation.ReLU'>'s flops has been counted\n",
      "Cannot find suitable count function for <class 'paddle.nn.layer.pooling.MaxPool2D'>. Treat it as zero FLOPs.\n",
      "<class 'paddle.nn.layer.pooling.AdaptiveAvgPool2D'>'s flops has been counted\n",
      "Cannot find suitable count function for <class 'paddle.fluid.dygraph.nn.Flatten'>. Treat it as zero FLOPs.\n",
      "<class 'paddle.nn.layer.common.Linear'>'s flops has been counted\n",
      "+-----------------------+-----------------+-----------------+---------+---------+\n",
      "|       Layer Name      |   Input Shape   |   Output Shape  |  Params |  Flops  |\n",
      "+-----------------------+-----------------+-----------------+---------+---------+\n",
      "|       conv2d_21       |  [1, 1, 32, 32] | [1, 64, 16, 16] |   3200  |  819200 |\n",
      "|     batch_norm2d_0    | [1, 64, 16, 16] | [1, 64, 16, 16] |   256   |  32768  |\n",
      "|        re_lu_0        | [1, 64, 16, 16] | [1, 64, 16, 16] |    0    |    0    |\n",
      "|      max_pool2d_2     | [1, 64, 16, 16] |  [1, 64, 8, 8]  |    0    |    0    |\n",
      "|       conv2d_22       |  [1, 64, 8, 8]  |  [1, 64, 8, 8]  |  36864  | 2359296 |\n",
      "|       conv2d_23       |  [1, 64, 8, 8]  |  [1, 64, 8, 8]  |  36864  | 2359296 |\n",
      "|     batch_norm2d_1    |  [1, 64, 8, 8]  |  [1, 64, 8, 8]  |   256   |   8192  |\n",
      "|     batch_norm2d_2    |  [1, 64, 8, 8]  |  [1, 64, 8, 8]  |   256   |   8192  |\n",
      "|       conv2d_24       |  [1, 64, 8, 8]  |  [1, 64, 8, 8]  |  36864  | 2359296 |\n",
      "|       conv2d_25       |  [1, 64, 8, 8]  |  [1, 64, 8, 8]  |  36864  | 2359296 |\n",
      "|     batch_norm2d_3    |  [1, 64, 8, 8]  |  [1, 64, 8, 8]  |   256   |   8192  |\n",
      "|     batch_norm2d_4    |  [1, 64, 8, 8]  |  [1, 64, 8, 8]  |   256   |   8192  |\n",
      "|       conv2d_26       |  [1, 64, 8, 8]  |  [1, 128, 4, 4] |  73728  | 1179648 |\n",
      "|       conv2d_27       |  [1, 128, 4, 4] |  [1, 128, 4, 4] |  147456 | 2359296 |\n",
      "|       conv2d_28       |  [1, 64, 8, 8]  |  [1, 128, 4, 4] |   8192  |  131072 |\n",
      "|     batch_norm2d_5    |  [1, 128, 4, 4] |  [1, 128, 4, 4] |   512   |   4096  |\n",
      "|     batch_norm2d_6    |  [1, 128, 4, 4] |  [1, 128, 4, 4] |   512   |   4096  |\n",
      "|     batch_norm2d_7    |  [1, 128, 4, 4] |  [1, 128, 4, 4] |   512   |   4096  |\n",
      "|       conv2d_29       |  [1, 128, 4, 4] |  [1, 128, 4, 4] |  147456 | 2359296 |\n",
      "|       conv2d_30       |  [1, 128, 4, 4] |  [1, 128, 4, 4] |  147456 | 2359296 |\n",
      "|     batch_norm2d_8    |  [1, 128, 4, 4] |  [1, 128, 4, 4] |   512   |   4096  |\n",
      "|     batch_norm2d_9    |  [1, 128, 4, 4] |  [1, 128, 4, 4] |   512   |   4096  |\n",
      "|       conv2d_31       |  [1, 128, 4, 4] |  [1, 256, 2, 2] |  294912 | 1179648 |\n",
      "|       conv2d_32       |  [1, 256, 2, 2] |  [1, 256, 2, 2] |  589824 | 2359296 |\n",
      "|       conv2d_33       |  [1, 128, 4, 4] |  [1, 256, 2, 2] |  32768  |  131072 |\n",
      "|    batch_norm2d_10    |  [1, 256, 2, 2] |  [1, 256, 2, 2] |   1024  |   2048  |\n",
      "|    batch_norm2d_11    |  [1, 256, 2, 2] |  [1, 256, 2, 2] |   1024  |   2048  |\n",
      "|    batch_norm2d_12    |  [1, 256, 2, 2] |  [1, 256, 2, 2] |   1024  |   2048  |\n",
      "|       conv2d_34       |  [1, 256, 2, 2] |  [1, 256, 2, 2] |  589824 | 2359296 |\n",
      "|       conv2d_35       |  [1, 256, 2, 2] |  [1, 256, 2, 2] |  589824 | 2359296 |\n",
      "|    batch_norm2d_13    |  [1, 256, 2, 2] |  [1, 256, 2, 2] |   1024  |   2048  |\n",
      "|    batch_norm2d_14    |  [1, 256, 2, 2] |  [1, 256, 2, 2] |   1024  |   2048  |\n",
      "|       conv2d_36       |  [1, 256, 2, 2] |  [1, 512, 1, 1] | 1179648 | 1179648 |\n",
      "|       conv2d_37       |  [1, 512, 1, 1] |  [1, 512, 1, 1] | 2359296 | 2359296 |\n",
      "|       conv2d_38       |  [1, 256, 2, 2] |  [1, 512, 1, 1] |  131072 |  131072 |\n",
      "|    batch_norm2d_15    |  [1, 512, 1, 1] |  [1, 512, 1, 1] |   2048  |   1024  |\n",
      "|    batch_norm2d_16    |  [1, 512, 1, 1] |  [1, 512, 1, 1] |   2048  |   1024  |\n",
      "|    batch_norm2d_17    |  [1, 512, 1, 1] |  [1, 512, 1, 1] |   2048  |   1024  |\n",
      "|       conv2d_39       |  [1, 512, 1, 1] |  [1, 512, 1, 1] | 2359296 | 2359296 |\n",
      "|       conv2d_40       |  [1, 512, 1, 1] |  [1, 512, 1, 1] | 2359296 | 2359296 |\n",
      "|    batch_norm2d_18    |  [1, 512, 1, 1] |  [1, 512, 1, 1] |   2048  |   1024  |\n",
      "|    batch_norm2d_19    |  [1, 512, 1, 1] |  [1, 512, 1, 1] |   2048  |   1024  |\n",
      "| adaptive_avg_pool2d_0 |  [1, 512, 1, 1] |  [1, 512, 1, 1] |    0    |   1024  |\n",
      "|       flatten_0       |  [1, 512, 1, 1] |     [1, 512]    |    0    |    0    |\n",
      "|       linear_10       |     [1, 512]    |     [1, 10]     |   5130  |   5120  |\n",
      "+-----------------------+-----------------+-----------------+---------+---------+\n",
      "Total Flops: 35529728     Total Params: 11185034\n",
      "35529728\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. \n",
      "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
      "  if data.dtype == np.object:\n"
     ]
    }
   ],
   "source": [
    "FLOPs = paddle.flops(model, (1, 1, 32, 32), print_detail=True)\n",
    "print(FLOPs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "为了验证残差连接对深层卷积神经网络的训练可以起到促进作用，接下来先使用ResNet18（use_residual设置为False）进行手写数字识别实验，再添加残差连接（use_residual设置为True），观察实验对比效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.4.2 没有残差连接的ResNet18\n",
    "\n",
    "为了验证残差连接的效果，先使用没有残差连接的ResNet18进行实验。\n",
    "\n",
    "#### 5.4.2.1 模型训练\n",
    "\n",
    "使用训练集和验证集进行模型训练，共训练5个epoch。在实验中，保存准确率最高的模型作为最佳模型。代码实现如下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/nn/layer/norm.py:653: UserWarning: When training, we now always track global mean and variance.\n",
      "  \"When training, we now always track global mean and variance.\")\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Train] epoch: 0/5, step: 0/80, loss: 2.53809\n",
      "[Train] epoch: 0/5, step: 15/80, loss: 1.82928\n",
      "[Evaluate]  dev score: 0.18500, dev loss: 2.20912\n",
      "[Evaluate] best accuracy performence has been updated: 0.00000 --> 0.18500\n",
      "[Train] epoch: 1/5, step: 30/80, loss: 0.98511\n",
      "[Evaluate]  dev score: 0.59000, dev loss: 1.35768\n",
      "[Evaluate] best accuracy performence has been updated: 0.18500 --> 0.59000\n",
      "[Train] epoch: 2/5, step: 45/80, loss: 0.62413\n",
      "[Evaluate]  dev score: 0.72000, dev loss: 0.81154\n",
      "[Evaluate] best accuracy performence has been updated: 0.59000 --> 0.72000\n",
      "[Train] epoch: 3/5, step: 60/80, loss: 0.33644\n",
      "[Evaluate]  dev score: 0.77000, dev loss: 0.68273\n",
      "[Evaluate] best accuracy performence has been updated: 0.72000 --> 0.77000\n",
      "[Train] epoch: 4/5, step: 75/80, loss: 0.16001\n",
      "[Evaluate]  dev score: 0.81000, dev loss: 0.60809\n",
      "[Evaluate] best accuracy performence has been updated: 0.77000 --> 0.81000\n",
      "[Evaluate]  dev score: 0.79000, dev loss: 0.62241\n",
      "[Train] Training done!\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAF6CAYAAAAtT8R+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3Xd4nNWZ/vHvM12jatmy5V7lggHb4AKmGIjBZiFAQmBJJ0tCIBXCJpSEbEihhRoC2c2mLZuEkIQEwm8Bg+nVYGywccMFNxl3W3U09fz+GFmxDbYlWZpXI92f65rLM++cmbkRlzTzzDnvc8w5h4iIiIiIiEi+8nkdQERERERERORwqLAVERERERGRvKbCVkRERERERPKaClsRERERERHJaypsRUREREREJK+psBUREREREZG8psJWRERERERE8poKWxEREREREclrKmxFREREREQkr6mwFRERERERkbwW8DrA4ejTp48bNmyY1zFERKSbePPNN7c75yq8zpHP9N4sIiIdqbXvzXld2A4bNoz58+d7HUNERLoJM1vndYZ8p/dmERHpSK19b9ZSZBEREREREclrKmxFREREREQkr6mwFRERERERkbymwlZERERERETymgpbERERERERyWt53RVZRKSjZTIZNm7cSENDg9dRpIMFg0H69u1LSUmJ11FERESkg6mwFRHZy/bt2zEzxowZg8+nRS3dhXOOWCxGdXU1gIpbERGRbkaf2kRE9rJ792769eunorabMTOi0SgDBw5k69atXscRERGRDqZPbiIie0mn0wSDQa9jSCcpKCggmUx6HUNEREQ6mApbEZH9mJnXEaST6P+tiIhI96TCVkRERERERPJajy9snXPEHl5GprbJ6ygiIl3W8uXLMTPmz59/WM/zxBNPYGZs3769g5KJiIhIW2ViCVJbanDpjNdROkyP74qcWraNnR97kKJrTqT0pplexxERaZdDLbEdOnQoa9eubffzV1VV8f7779OnT592P4eIiIjknktnyNTG8PcqBKDx+eWk1mwDwCJBgqMrCY2pxFcU8TLmYevxhW3wiL4UfG4C9Xe8SuGlxxIY3svrSCIibfb++++3XH/llVc4//zzWbBgAf379wfA7/d/6OMSiQShUOiQz+/3+6msrOyYsNJlmNls4G7AD/zKOXfzfvcPAf4HKGsec41z7rGcBxURkVbLxBKkN9eQ3laXveyogwwUf+Z4LOAnOLgcf+8ifNEwyTVbSSzeQGZXA9GZ44HsitZ87EmRk6XIZjbYzJ41s6VmtsTMvvkhY04xsxoze6v58v1cZAMovfEjWMBH7dVP5eolRUQ6VGVlZculvLwcgIqKipZjFRUVLeNuuOEGLr30UsrLyzn99NMBuO222zj66KMpLCxkwIABfOYzn9lnW5z9lyLvuf23v/2NM888k2g0yqhRo/jjH//Y5uwvvfQSJ554IpFIhPLycj73uc+xY8eOlvvXrVvHeeedR+/evSkoKGDUqFHcfffdLff/9a9/ZcKECUSjUXr16sXxxx/PO++80/YfYg9jZn7gXuBM4Ajgk2Z2xH7Dvgf82Tk3CbgIuC+3KUVE5GBcKk1qSw3xxRvJxBIAJNdsI/bcchLLNwEQGjuAglPGANliNTiiL+EjBxEcUUF05niKPjGF8OThAGTqmqh/aD7xRRtani9f5GrGNgVc5ZxbYGbFwJtm9pRzbul+4150zp2do0wt/ANLKLrmROq+/yzxF9cRPmloriOIiOTM7bffzjXXXMO8efNIp9NAdinzXXfdxfDhw9m0aRNXXnkln/3sZ5kzZ85Bn+vqq6/mlltu4Z577uE///M/ufjii5k+fTrDhg1rVZYNGzYwa9YsLrjgAv7rv/6L7du3c9lll3HRRRfx1FPZLxu/9KUv4ff7eeaZZygtLWX16tUthe/69eu56KKLuP322znnnHOIxWK8+eabB5yhln1MBVY559YAmNmfgHOBvd+bHVDSfL0U2JTThCIi8gGZuibiSzaS3lpHZmcDOAeAryyKb3A5weF9CPQrwderEPMfeh5z7yXILpnCVxgm/uZa4gvXERjWh9DY/vj7lnT5WdycFLbOufeB95uv15nZMmAg+755eqroquNp/OWb1FzxBBVvfAnz9fi+WiIC7L7icZJvbc756wYnVlJ215md8twnnXQS11133T7Hrrrqqpbrw4cP5+6772b69Ons2LGD3r17H/C5rrzySj7+8Y8DcOONN3LPPffw/PPPt7qw/dnPfka/fv341a9+RSCQfUv63e9+x3HHHcfrr7/O1KlTWbduHV/4wheYMGECwD7PXV1dTSaT4cILL2xZdn3EEftPOsoBDAQ27HV7IzBtvzE/AJ40s68DhYCaUYiI5IhLpkhvqye9rZb0tjqCwysIjuyLyziSK7fg71NM6KhB+CuK8VcU4yvInlrki4YhGm7Xa/rLiyg882jSuxtILt9MYtUWUmu3U/SvU7FIqEsvU8559WZmw4BJwLwPuft4M3vbzB43s/G5zOWLhii5ZSbJBe/TeP/buXxpEZGcmjp16geOzZ07l9NPP53BgwdTXFzMzJnZ+mXdunUHfa6JEye2XA+FQvTp04ctW7a0OsuSJUuYPn16S1G7J18kEmHJkiUAfOtb3+L666/n+OOP59prr+Xll19uGTtlyhRmzJjBmDFjOP/887nnnnuorq5u9evLIX0S+J1zbhDwL8D/mtkHPjuY2aVmNt/M5m/bti3nIUVE8p1zDhdPZq+nM9Q/vIC6P7xK45zFxBesI1Mba+lg7CuJUPzp6RSeeTSRY4cRHNK7pajtKP6yQiLHjaT4X6cRPX08vkj2+WNzlxJ7dRXpnQ0d+nodIafNo8ysCHgIuMI5V7vf3QuAoc65ejP7F+BhoOpDnuNS4FKAIUOGdGi+gk8eRf09r1N73dMUfOIIfEXt+6ZDRLqPzpo19VJhYeE+t1etWsXZZ5/NF7/4RW644QZ69+7N6tWrOeuss0gkDn5+zf6Np8yMTKZjtw748pe/zFlnncUTTzzBs88+y+mnn86nPvWpllneZ555hnnz5jF37lz+9Kc/cfXVV/PII4+0nD8sB1QNDN7r9qDmY3u7BJgN4Jx71cwiQB9g696DnHO/BH4JMHnyZNdZgUVEugsXT5La09xpay3p7XUE+pUSnTke8/vw9ykiMLQ3gb4l+PsUYeFgy2PNbM/psp3Ogn4CA7LNdV0mg0UCJFduJrn8ffz9SgiN7U9gaJ9WLXnubDlLYGZBskXtH5xzf9v/fudcrXOuvvn6Y0DQzD6wr4Rz7pfOucnOucl7mqF0YEbK7pxF5v166m95+dAPEBHpBubNm0cymeSuu+5i+vTpjBkzhs2bc7P8evz48bzyyiukUqmWY6+//jpNTU0ceeSRLccGDRrEF7/4Rf7whz9w33338Zvf/IZ4PA5k/3Yfd9xxfO973+Pll19m6tSp/O53v8tJ/jz3BlBlZsPNLES2OdQ/9huzHvgIgJmNAyKApmRFRNrAZRzpnfUk1/5zD/fGuUuJPbWExNvrcbEkwWHZZcZ7FJw4msikoQQG9tqnqPWS+XwUnDSGogunEZ4ynExjgtjzK0i+m/tTtj5MTmZsLbsQ+9fAMufcHQcYUwlscc45M5tKtuje8WFjO1PouMEUfOoo6m57hegXjyEwtCzXEUREcmr06NFkMhnuvPNOPvGJT7BgwQJuuummnLz2N7/5Te677z6++MUv8u1vf5vt27dz+eWXM3PmTKZMmQLAZZddxnnnnUdVVRWxWIyHH36YkSNHEg6Hee6553jllVeYOXMmlZWVLF++nKVLl2q2thWccykz+xowh+xWPr9xzi0xsx8C851z/wCuAv7bzK4k20jqYuecZmRFRA4hta2W1Lod2RnZ7XWQyoDPCAyejvl9hCcNAbPsbGwwv3Zg9UWChI8cRGj8QNKbduOvKAYg8e5mUuu2Exw7IFuQ+3J7Lm6ufoonAJ8FFpvZW83HrgOGADjn/hP4BHC5maWAGHCRV2+eJTfPpOnvy6i9Zi7lD3zCiwgiIjkzZcoU7rjjDm6//Xa+//3vM23aNO68804++tGPdvprDxo0iDlz5nD11Vdz7LHHEo1GOfvss7nzzjtbxqTTab7+9a+zceNGotEo06dP59FHHwWgV69evPDCC9x9993s3r2b/v37c8kll3D11Vd3evbuoHmF1GP7Hfv+XteXkn0PFxGRD+HSGTI7G0g1N3iKTBuBLxIi/X4NiXeq8fUuJFRVmW3w1LcYmou9Pct785mZERi413+Hc6R31JOau4TAiAqiM8bmNk8+f/E6efJkt2dPxY5W+x/PUvfD5+nz8r8Rnt6x5/KKSNe1bNkyxo0b53UM6UQH+39sZm865ybnOFK30pnvzSIiXnLOgQPzGeltdTS9vob0jjpIZ+spi4aIfuQI/H2KcYkU+AwL9Kzt51wmQ2rdDqwgSKCyY1a+tva9Ob/mvXOo6Dsn0PCrBdRcOYeKVy/R9j8iIiIiIj2IS6VJ76gnvbWuZcud8DFDCVVVQjBbsIbGDsDftxh/RQm+wn82nrVQzyyzzOcjOLxj+yC1Vs/8ibeCrzBE6U0fYdfnHyb2x8VEPzPB60giIiIiItIJnHO4uiZcxuEvi+LiSeoemAfNq1utOEKgshRfUQQAf1mUwrNUH3QlKmwPouAzR1N/z+vUXDOXyMfG4Svs2P2hRERERETEG6nNu0lvqW3ZcsfFUwSG9Cb6kSOwcJDwMUPxlUXxVxR3+D6x0vG0vvYgzOej9K7ZZKrrqL/tFa/jiIiIiIhIGznnSO9uIPHuZuKLN7Qcb3pjLfEF68jUxggM6U1k+ijCxw5tuT989GCCQ3qrqM0TmrE9hPAJQyi4cDz1t7xE4SWT8A8q9TqSiIiIiIgcQnL1VhKrtmS320mkgWyDp9CRgzAzCk4ejS8S7DL7xMrhUWHbCiW3zCT2yHJqrn2a8v/9uNdxREREREQEcBlHZldDdjlxc4OnwrMmYOEgmYY4rilJcHgF/ooS/H2L8ZUUYJbdcsdfGvU4vXQkFbatEBjWi6KrplN/44skvj6V0NRBXkcSEREREelxMo0JLOjDggGS63YQe2E5pDIAWCSIv6IYl0hj4SChowYRPnqwx4klV1TYtlLxNSfS+JuF1FzxBH1evqTlmx4REREREel4zjkyOxtIbalp2XLH1ceJnDyG0Mi++MoKCFVV4q8oxt+3BCsK7/MZXZ/XexY1j2olX3GYkp+cRuLVjcQefMfrOCIiIiIi3U6mMU66phEA1xCn4R8Lic9bQ3pLDf4+xYSnDCdQUQxklxJHjhtJcGRffMURFbI9nGZs2yD6+Yk0/Px1ar/zFAXnjsUKdKK5iMjeLr74YjZu3MjcuXO9jiIiInnApdKkt9SSqt5FqnoXmd2NBIb2JnraEfiKIhScOi673U5h2Ouo0sVpxrYNzO+j9M7ZpDfUUn/Hq17HERHZx8UXX4yZYWYEg0H69OnDiSeeyK233kpDQ4PX8URERLLLi+ubWm43PL6YxiffIbFsE1YQIjx5GOGJ/9xyJzisj4paaRXN2LZReMYwIuePo+6mF4l+YSL+ASVeRxIRaXHSSSfx5z//mUwmw44dO3jppZe46aab+PWvf80LL7xAv379vI4oIiI9TCaWIL1pN6lNu0hV78YlUxR/6njM7yM8cTBmhr+yFAv4vY4qeUwztu1QeuvpuGSG2u8+43UUEZF9hEIhKisrGTBgAEcddRSXX345r776Ktu2beOaa67ZZ+w999zD2LFjiUQiVFVV8ZOf/IRUKgXAd7/7XcaMGfOB57/88ss58cQTW53HOcdtt93GiBEjCIVCjBw5krvuumufMY888giTJk0iGo1SVlbG1KlTWbhwIQDJZJJvfetbDBo0iHA4TP/+/bnooova+mMREZEccukMLpPtVJxYuon6P80j9sIKUht24q8sITJtJDgHQHBwbwKDylXUymHTjG07BEaUU3TFcdTf+jKFX5tK6NgBXkcSETmggQMH8ulPf5r777+fX//61/h8Pn7wgx/w29/+lrvuuouJEyeybNkyLrvsMpqamvjRj37E5z//eW688UbmzZvHtGnTAIjH4zz44IPcfPPNrX7t++67j+uvv567776bU089laeffporrriC4uJiLrnkEjZv3swFF1zAj3/8Yy644AKamppYuHAhgUD27emee+7hz3/+M7///e8ZMWIEW7Zs4eWXX+6Un5OIiLSPc45MTYxU9S7Sm3aR2lxD9NRxBAaV468sIXzMUAIDe+ErL8J8avAknUOFbTsVf/ckGn/3FjVXPkGf57+gLmwi3VjD44s+cCw4rA+hcQNwqTSNTy354P2j+hGq6kemKUns2WUfuD80pj/BERVk6uPEXlzxgfsLzzy6Y8I3Gz9+PLW1tWzfvp2ioiJuvfVW/va3vzF79mwAhg8fzo9//GO+8Y1v8KMf/YjRo0czbdo07r///pbC9tFHHyUWi3HhhRe2+nVvvvlmvv71r3PppZcCUFVVxYoVK/jJT37CJZdcwvvvv08ymeTCCy9k2LBhAIwbN67l8evWrWP06NHMmDEDM2PIkCFMmTKlg34qIiLSXs45zIxMfZyGx97GNcQB8JUUEKzqhzWfF+svL8JfXuRlVOkhtBS5nXwlEUp+fBqJF9fT9NBSr+OIiByUa17yZWYsWbKEWCzG+eefT1FRUcvly1/+MjU1NWzbtg2Az3/+8zz44IMkk0kA7r//fs455xzKyspa9Zq1tbVs3LiRk08+eZ/jM2bMYO3atTQ2NnL00Ucza9YsjjzySD72sY9x9913s2HDhpaxX/jCF1i8eDGjRo3isssu46GHHiKRSHTEj0RERNrAZTKkNtfQtGAt9Y++RdO81QBYYYhAZSmR6aMo+sQUis6fTMFxo/D3KvQ4sfQ0mrE9DNF/m0T9z1+n5ttPETl7NBbR9j8i3dHBZk8t4D/o/b5I8OD3F4U7fHb2wyxZsoTS0lJ69+7NmjVrAPjLX/7C6NGjPzC2vLwcgIsuuogrrriC//u//+OEE07giSee4OGHH+7QXH6/n8cff5w33niDuXPn8tBDD3HNNdfwl7/8hbPPPpuJEyfy3nvv8dRTT/Hss8/yzW9+k+uvv57XXnuNkhI17xMRyYXYyytJvrcNkmkw8FeU4C+NAtkvTAtO/mBPBpFc04ztYTC/j7I7Z5Feu5v6u17zOo6IyIeqrq7mD3/4Ax//+Mfx+XyMHz+eSCTCmjVrGDVq1Acufn+2gUevXr346Ec/yv/+7//ywAMPUF5ezqxZs1r9uiUlJQwaNIgXXnhhn+PPP/88w4cPJxr954eiqVOnct111/HCCy8wY8YMfvvb37aMLyoq4mMf+xg/+9nPmD9/PsuWLeP555/vgJ+MiIjszSVSJNdtJ/bKyuzy4j2rfSJBgiMqKDh1HMWfPJ7CsyYQGqceM9K1aMb2MIVPG0Hk3DHU3fgi0Ysn4q8s9jqSiPRgiUSCzZs3f2C7n759+3LTTTcB2ULxuuuu47rrrsPMmDlzJqlUisWLF7Nw4UJuueWWluf73Oc+xwUXXMCyZcv49Kc/3VL0tta1117LVVddRVVVFaeccgrPPPMMv/jFL7j33nsBeOWVV3j66ac544wz6N+/PytXrmTRokVccsklAPz0pz9lwIABTJw4kWg0ygMPPIDf7//QmWYREWmf5IadJBZtIL2tFhwQ9BPoX5adoQ0FiBw7zOuIIoekwrYDlP70DLaMv5fa65+l13+f43UcEenBXnzxRfr374/f76e0tJRx48bxta99ja9+9asUFv7zfKfrr7+e/v378/Of/5yrrrqKgoICRo8ezcUXX7zP85155pmUlpaybNkyHnjggTbnufzyy2loaODGG2/kK1/5CoMHD+bmm29uKVxLS0t59dVXuffee9m1axeVlZV8+tOf5vrrrweys7533HEHK1euJJPJMG7cOB566KEP3YpIREQOLVPX1Lyf7C7CE4dkGztlHC6TIXT0YAIDeuHvW4z5tLBT8ovtWWKQjyZPnuzmz5/vdQwAaq6aQ/2dr1Kx4MuEJvb3Oo6ItNOyZcv26cor3c/B/h+b2ZvOuck5jtStdKX3ZhHJyjQliL+1nnT1bjK1MQAsGiIyvYrg4HKP04kcXGvfm/VVTAcpvv5kfOUF1Fw5h3z+skBERERE8pdzjvT2OuJvryexagsA5veTXL0NX3GE8NQRFH7sWIounKqiVroVLUXuIL6yAop/dBo1X/k/mh5ZTsF5mvERERERkdxIrtlKcv1O0pt24eIpILunOqP6YUE/xZ88DvOZxylFOo8K2w5U+KVjaLj3dWr+/UkiZ1ZhYf14RURERKRjuVSa9OYa0rsaCR81CIDk6q2kd9QTGFROYGAv/APK8BWEWh6jola6O1VeHcgCfkrvmMWOWb+n/p55FP/7CV5HEhEREZFuIFMbI7luO6nq3aS31EDGgd9HaEwlFgpQcPJYCPkxUwErPZMK2w4WOWMU4bOqqPvRC0Q/PxF/ReGhHyQiIiIispdMLEFq024C/cvwRUOkNu0mPn8tvl5RQuMGZGdl+5Vggew2bFopKD2dfgM6QeltZ7D1qF9Q9/1nKfvF2V7HEZE2cs7pG+9uSs39RKSrcukM6S01pKp3k9q0i8zOBgAiJ44mVNWP4PAKAkPK8UXDHicV6ZrUFbkTBMdWUPiVKTT88k2Si7d4HUdE2sDv95NMJr2OIZ0kFosRDAa9jiEiku1evKuB9M767O14ksY575BYWo2FAoSPHUbhRycRHNUXyM7IqqgVOTDN2HaSkv+YQeP/vk3Nt+bQ+8nPavZHJE+UlZWxZcsWBg4ciE+b03cbzjlisRjV1dX069fP6zgi0kNlmpKkN+0itWk3qepduMYEgcHlRGeOxxcNE511FP6KYizo9zqqSN5RYdtJfOVRSm44lZpvPE7T/71LwdljvI4kIq3Qp08fNm7cyIoVK7yOIh0sGAzSr18/SkpKvI4iIj2ES2fI1MTwl2d7rjQ+9Q6Z7fUQChAYUEZgYC8CA8paxu99XUTaRoVtJyq8bDIN971B7VVPEjljJBbSj1ukq/P5fAwZMsTrGCIikoecc2RqY6Srm2dl398NDoo/dTwW8BGZPBwL+PH1LtL2OyIdTOvsOpEF/ZTefgapd3fQcN8bXscRERERkU6UWFJNw9/epGneGtK7GwmO6kfBjDHQXMMG+pdllxqrqBXpcJpC7GThM6sIzxpJ7Q3PU/DZCfh7R72OJCIiIiIdJLlhBxYJEqgoITCoHAv4CAzoha+kwOtoIj2KZmw7mZlRevssXF2cuh8853WcNknXxEhu2Ol1DBEREZEux6XSxF5dRWzuUhKLNgLgL4sSGjtARa2IB1TY5kBwfF8KvzyZhl+8QXLpVq/jtFp83mpiLywnE0t4HUVERESky0jvrKfh0bdILn+f0PiBFJwy1utIIj2eCtscKb7hFKwoRM1VT3odpdXCU0dAKkP8jfe8jiIiIiLSJaS31dHw6Fu4eJLoGUcSmToC8+sjtYjX9FuYI/4+hZT8xynEn1hF0+MrvY7TKv6yKKEjB5FcvZXU5t1exxERERHxjHMOAF/vIkJHDaLwvGMJDOzlcSoR2UOFbQ4VfnUK/qpyaq6ag0umvY7TKuEJg7GiME2vrMKlM17HEREREcm55PodNDy8gEwsgfmMyDHD8EWCXscSkb2osM0hCwUove0MUsu20/Bf872O0yoW8BM5blT2G8nmbypFRKR7MLPZZrbCzFaZ2TUfcv+dZvZW8+VdM9PyHelRXCpN7JWVxJ5eCj6DPJmYEOmJtN1PjkU+OobwR4ZT+x/PEf3UUfjKu/72P8HB5QQHl3sdQ0REOpCZ+YF7gdOBjcAbZvYP59zSPWOcc1fuNf7rwKScBxXxSHpHPbHnl5OpiREaP5DwscN0Lq1IF6bfzhwzM0rvmIXb3UTtD5/3Ok6bpDbX0PT6Gq9jiIhIx5gKrHLOrXHOJYA/AeceZPwngQdykkykC4gv3oBLpNUgSiRP6DfUA8GjK4l+6Rga7n2D5IrtXsdptfTWWhJLqklu2OF1FBEROXwDgQ173d7YfOwDzGwoMBx4Jge5RDyTaYyTqWsCoOD4URSed4waRInkCRW2Hin54alYNEjtv+fP9j+h8QPxlUVpem01LqVzTEREepCLgL865z70j7+ZXWpm881s/rZt23IcTaRjJNdlG0TFXnoXAAsH1SBKJI+osPWIv28Rxd87mab/9y5NT632Ok6rmN9H5PiRuPo48bc3HPoBIiLSlVUDg/e6Paj52Ie5iIMsQ3bO/dI5N9k5N7mioqIDI4p0vpYGUc8sxQrDRI4f5XUkEWkHFbYeKvrGNPwjelHzrTl5MwMaqCwjOLIviXc2kq5p9DqOiIi03xtAlZkNN7MQ2eL1H/sPMrOxQC/g1RznE+l0mboYDf9YSHLFZkJHDqLw7In4y7p+Y08R+SAVth6ycPP2P+9spfFXC7yO02rhKcOJTBmBr7jA6ygiItJOzrkU8DVgDrAM+LNzbomZ/dDMztlr6EXAn5zTnm/S/VhBCIuGic46ksiU4WoQJZLHtN2PxyLnjSU0Yyi11z9LwUVH4ivr+sWiryBE6IgBADjnMDOPE4mISHs45x4DHtvv2Pf3u/2DXGYS6WyZhjjxt9ZnOx0H/RTOPsrrSCLSAfS1lMfMjNI7Z5PZ0Ujdj1/wOk6bJDfspOGRhbhEyusoIiIiIoeUXLedhkcWkFyzlfSOeq/jiEgHUmHbBYQm9Sf6b5Oo/9k8UqvyZysdX0GQzK4G4gvWeR1FRERE5IBcMk3s5ZXEnlmGFUUoPGcSgcpSr2OJSAdSYdtFlPz4NCwcoObbT3kdpdX8fYoJju1PYvkm0tv1raeIiIh0TU2vriL5bnODqLMm4C9VgyiR7kaFbRfhryym+LqTaHp4OfFn1ngdp9UixwzDIkFir67EZdRXRERERLoG5xwumd11IjxpCNFZR6lBlEg3pt/sLqToyuPwDy1l95VzcOmM13FaxcIBIlNGkNleT6p6p9dxRERERMg0xGmcs5jY88txzuE5/zISAAAgAElEQVQrLiAwoMzrWCLSiVTYdiEWCVL60zNILdpC428Xeh2n1QIjKoieeTTBwb29jiIiIiI93J4GUeltdQSG6LOJSE+hwraLiXziCEInDqH2u8+QqW3yOk6rmFlLA4ZMLOFxGhEREemJ9mkQVRyh8JxjCI2u1LaEIj2ECtsuJrv9zywyWxuou/FFr+O0SWrTLur/8gapzbu9jiIiIiI9jEulSW3cSejowc0Nogq8jiQiOaTCtgsKTR5I9PMTqL/zNVJr8ue8VX/fEqwgSNOrq3GZ/DhHWERERPKXyzgSK7fgMg5fQYiijx1L5NhhmE8fcUV6Gv3Wd1ElN34EC/iouXqu11FazQJ+ItNGktndSGLJJq/jiIiISDe2p0FU00vvklq3HQALBTxOJSJeUWHbRfkHlFB0zYk0/XUp8RfWeh2n1YJDehMY0pv4W+vI1Me9jiMiIiLdUHLtduofXkB6ex2RE6sIDOvjdSQR8VhOClszG2xmz5rZUjNbYmbf/JAxZmY/M7NVZrbIzI7JRbaurOiq4/EPLqHmiifyZvsfgMi0EWBG6n2daysiIiIdq2nhOmLPLsNXEqHo3GMIValBlIjkbsY2BVzlnDsCOA74qpkdsd+YM4Gq5sulwC9ylK3L8kVDlNxyOsmFm2m8/22v47SaryhC8QVTCFX18zqKiIiIdDPBQeUtDaJ8JWoQJSJZOSlsnXPvO+cWNF+vA5YBA/cbdi5wv8t6DSgzs/65yNeVFVx0JMHjBlF73dN5tbTXwkEAUltqcKm0x2lEREQkX7mMI/72eprmrQbAX1GsBlEi8gE5/4tgZsOAScC8/e4aCGzY6/ZGPlj89jhmRtlds8lsrqf+5pe8jtMm6d2NND62iPiiDYceLCIiIrKfTH0TjU8sIr5gHZlYApdxXkcSkS4qp4WtmRUBDwFXOOdq2/kcl5rZfDObv23bto4N2EWFpg2i4NNHUXfbK6TW5c95q/6yKMGRfUks3ki6ptHrOCIiIpJHku9to/6RhaR3NBA5cTQFM8ZiPp1LKyIfLmeFrZkFyRa1f3DO/e1DhlQDg/e6Paj52D6cc790zk12zk2uqKjonLBdUMlNMzGfUXv1U15HaZPwlOEQ8GX3tnX6llVEREQOLRNLEHvpXfylBRSdO4lQVT81iBKRg8pVV2QDfg0sc87dcYBh/wA+19wd+Tigxjn3fi7y5YPA4FKKvn0CsQeXEH9lvddxWs1XECJyzDDS7+8m9V7PmGEXERGR9knXNOKcw1cQovDMo4n+y9FqECUirZKrGdsTgM8Cp5nZW82XfzGzy8zssuYxjwFrgFXAfwNfyVG2vFH0nRPwDSjObv+TyZ/tf4Jj+uPvX4pL5U9mERERyZ09DaIa/v4myTXZL8L9fYrVIEpEWi2Qixdxzr0EHHT9iMuuU/1qLvLkK19hiNKbZ7Lrc38n9ofFRD87wetIrWI+IzrrKC0hEhERkQ/I1DcRe2EF6S21BIZXEBxU7nUkEclD+hoszxR8+iiCUwZQc+1cMg0Jr+O0mpnhnCOxagvpHfVexxEREZEuILluO/WPLCC9s4HISaMpmDEGC+dk3kVEuhkVtnnGfD5K75xNprqO+p++7HWctkmmib/xHrFXVqmRlIiIiIAZ/tJotkHUKDWIEpH2U2Gbh8InDKHgX8dTf+vLpDbUeB2n1SwUIDx1BJntdSTf3ex1HBEREfFAamstiRXZ/qDBIb2JnjUBX7EaRInI4VFhm6dKbjkdl3HUXjvX6yhtEhxRgb+ylKb5a8nE8mcptYiIiBwel3HE31pH42Nvk3inGpfONpXULK2IdAQVtnkqMLSM4n+fTuwPi0nM2+h1nFYzMyLHj4JUmvj8tV7HERERkRzI1DXR+Pgi4gvXExheQeFHJ2J+fQwVkY6jvyh5rOiaE/FVFrH7iify6pxVf1mU8OThBIb09jqKiIiIdDKXSNHw6ELSuxooOHkM0RljsZAaRIlIx1Jhm8d8RWFKbvwIydc2EvvTO17HaZPw+IEEh6qwFRER6a5alhqHAoSnjKDo3GMIjuzrcSoR6a5U2Oa56OcnEJxUSe3VT5FpzK9zVp1zxN9aT3xptddRREREpAOlttRS//c3SVXvAiBU1Q9fccTjVCLSnamwzXPm81F612zSG2qpv+NVr+O0iZmR3l5H/M21ZOrjXscRERGRw+QyjvjCdTQ+/jY4IOj3OpKI9BAqbLuB8MnDiJw/jvqbXiK9qdbrOG0SmTYSHDS9vtrrKCIiInIYWhpEvbWe4Ii+FJ07iUDfEq9jiUgPocK2myi99XRcKkPtdU97HaVNfMURwhOHkFq3g+SGnV7HERERkXZKVe/KNoiaMYaCk8eoQZSI5JQK224iMKKcoiuPo/F/3iYxP7/OWQ2NH4ivtID462twmfzp7iwiItLTuUSK1JYaAIJjKin6+GSCI9QgSkRyT4VtN1J83Un4+hZSc+WcvNr+x/w+IieNpmDGWMynTdpFRETyQWpLDfWPLCD29FJcMoWZ4YuGvI4lIj2UCttuxFcSoeTHp5F4aT1Nf13qdZw2CVSU4O9TBPxzewARERHpelzG0bRwHY2PLwIzCmaOx4Jadiwi3lJh281E/20SgaP7UfPtJ3FNSa/jtFnTvNU0zl2SVzPOIiIiPYVLpWl87G0Sb60nOLIvReeoQZSIdA0qbLsZ8/sou2s26XU11N/5mtdx2sxXUkB6025S723zOoqIiIjsxwJ+/H1LKJgxloKT1CBKRLoOFbbdUPjU4UTOHUPdjS+S3lzndZw2CY7pj693EU2vr8ElUl7HERER6fFcPEXshRWkd9QDEJk6guCICo9TiYjsS4VtN1X60zNw8RS133vG6yhtYj6j4PhRuFiS+MJ1XscRERHp0VKbsw2ikmu2thS2IiJdkQrbbipQ1Zuib0yj8TcLSSx83+s4beKvKCY4tj+JVVtw8fw7T1hEJF+Y2WwzW2Fmq8zsmgOMudDMlprZEjP7Y64zijdcJkPTgrU0PrEIfEb0rAmERld6HUtE5IBU2HZjxd87GV/vKDVXPpF3zZgixwyj6LxjsXDQ6ygiIt2SmfmBe4EzgSOAT5rZEfuNqQKuBU5wzo0Hrsh5UPFEcsVmEm9vIDiyH0XnTiJQoQZRItK1qbDtxnxlBRT/8FQSz6+j6eHlXsdpEwsH8BWGcc6RqY15HUdEpDuaCqxyzq1xziWAPwHn7jfmS8C9zrldAM65rTnOKDnknCPTmAAgOKaSgtPHU3DSaG3lIyJ5QYVtN1f4pWMIjK+g5t+fxMXzrxlTfP5a6h99i0xTwusoIiLdzUBgw163NzYf29toYLSZvWxmr5nZ7Jylk5xy8SSx55bT8OhCXDyJ+XwEB5V7HUtEpNVU2HZzFvBTeuds0mt2Uf+zeV7HabNgVV9IponPX+t1FBGRnigAVAGnAJ8E/tvMyvYfZGaXmtl8M5u/bZu2a8s32QZRC0mt20FobH/QDK2I5CEVtj1A5PSRRM4eTd2PXyC9Nb86GvrLCgkdOZDkyi2kttR4HUdEpDupBgbvdXtQ87G9bQT+4ZxLOufeA94lW+juwzn3S+fcZOfc5IoKbQOTL1zG0fTmWhofX4T5jMKzJhCeMATzmdfRRETaTIVtD1Fy2xm4xiS133/W6yhtFp4wBCsM0/TKKlwm43UcEZHu4g2gysyGm1kIuAj4x35jHiY7W4uZ9SG7NHlNLkNKJzJI76gnWNWPwnOPwV9R7HUiEZF2U2HbQwTH9KHwq1No/O8FJBdt9jpOm1jQT2TaSFwiRaauyes4IiLdgnMuBXwNmAMsA/7snFtiZj80s3Oah80BdpjZUuBZ4NvOuR3eJJaO4JwjsXILmfomzIzoR46g4MTRWNDvdTQRkcNi+bYNzN4mT57s5s+f73WMvJHZ2cjmqnsITaqk91Ofwyy/lhq5VBoL6I1XRDqPmb3pnJvsdY58pvfmrq1pwVoSb28gdORAIlNGeB1HROSQWvverBnbHsRXHqXkB6cQf/o9mv7fu17HaTML+HHpDMn31JhERESkrTJNSRLvVBMY1ofwscO9jiMi0qFU2PYwhZdNJjC2DzVXzcEl8m/7n+SKzcSeW05y406vo4iIiOSV5Ir3IZ0hPFENokSk+1Fh28NY0E/pHbNIr9xJw71veB2nzYJjKvGVFtD02mpcKu11HBERkbzg0hkSyzbhH9gLf69Cr+OIiHQ4FbY9UOTMKsKzR1F7w3Oktzd4HadNzO8jctwoXF0T8cUbvY4jIiKSHzKO4Jj+hI8e5HUSEZFOocK2hyq9/QxcfYK6HzzndZQ2CwwoIzCigsSiDaRrGr2OIyIi0uVZ0E9k0lAClWVeRxER6RQqbHuo4BF9KbxsMg3/OZ/kkq1ex2mzyJQR+PuVQiZ/u3qLiIjkQmprLcl123F6zxSRbkyFbQ9W/INTsOIwNd+aQ75t++SLhiicfZTOExIRETmE+MJ1NL22GvLsvV5EpC1U2PZg/j6FlPzHDOJPrib++Eqv47RLpilJ0+tr8rLDs4iISGdL76wnvWk3oXEDML8+9olI96W/cD1c4Vem4K8qp+aqJ3HJ/Osy7OqaSCypJr5wnddRREREupzEO9UQ8BEaU+l1FBGRTqXCtoezUIDS22eRWr6dhv+c73WcNvNXFBMcU0li2SbSO+q9jiMiItJlZBrjJN/bRqiqEgsHvY4jItKpVNgKkbNHE545gtr/eJbMzvzrMhw5dhgWDtL06qq8O1dYRESks2Tq4/iKIoTGD/Q6iohIp1NhK5gZpXfMwtXEqb3hea/jtJmFg4SnDCe9rY7kyi1exxEREekSAn1LKPz4sfiKI15HERHpdCpsBYDgUf2IfukYGu57g+TybV7HabPgyL6EJg4hMKCX11FEREQ8l97VgEulMTOvo4iI5IQKW2lR8sNTsWiQmn9/0usobWZmRCYNxVcU9jqKiIiIp1zG0Th3KbHnlnsdRUQkZ1TYSgt/3yKKrz+Z+P+tpOnJVV7HaZdMY5yGOYtJbanxOoqIiIgnUuu34+qbCFb18zqKiEjOqLCVfRR9fRr+kb2o+dYcXCr/tv+xYIBMTYymV1bhMhmv44iIiOSUc47EO9VYcYTA4N5exxERyRkVtrIPCwco/ekZpJZso+G/F3gdp80s6CcybQSZ3Y0klm7yOo6IiEhOpbfWkt5WR3j8QMyn82tFpOdQYSsfEDlvLKFThlF3/TNkdse8jtNmgSG9CQwuJ75wPZmGuNdxREREcia1fieEAgRHaRmyiPQsKmzlA8yM0jtnkdkZo+5HL3gdp83MjMi0keAc8bfWex1HREQkZ8KTh1F07jFY0O91FBGRnFJhKx8qNLE/0X+bRP0980it3OF1nDbzFUeIzjyCyNThXkcRERHJCZfOYGbaIUBEeiQVtnJAJT8+DQsHqPl2/m3/AxAY0AsLBnDpDC6tRlIiItJ9ZZqS1D84j8TqrV5HERHxhApbOSB/ZTHF3z2JpkdW0PT0Gq/jtItLpmh4ZAHxRRu8jiIiItJpkivex8VT+MsLvY4iIuIJFbZyUEVXHId/WFl2+588nPW0YABf7yISizaQrsm/RlgiIiKH4tIZEss24R/YC38vFbYi0jOpsJWDskiQ0ltPJ7VoC42/Weh1nHaJTBkBfh9Nr63COed1HBERkQ6VXLMVF0sSHj/Q6ygiIp5RYSuHFPnEEYROHELtd58mU9PkdZw280VDRI4ZSnrTblJrt3sdR0REpMM450gsqcbXqxD/gDKv44iIeEaFrRySmVF612wy2xupu/FFr+O0S3DsAHzlhSTVVENERLoRM6PglHFEjh+JmXkdR0TEMypspVVCxw4g+vmJ1N/1Gqk1O72O02bmM6KnH0nBaUd4HUVERKRD+cuiBPqVeh1DRMRTKmyl1Up+choW9FHznae8jtIuvmgI8xmZpiSZOjWSEhGR/JbeWU/js8vI1OffaUIiIh1Nha20mn9ACUXXnEjTQ8uIP7/W6zjt4pyj8bG3iT2/Qo2kREQkryXeqSa1cScW9HsdRUTEcypspU2Kr5qOf3AJNVc+kZ/b/5gROmow6W11JFdu8TqOiIhIu2Qa4iTXbCNUVYmFg17HERHxnApbaRMrCFJyy+kkF26m8X/e8jpOuwRH9cXfr4T4/PfINCW9jiMi0m6W9SUze8bMFjUfO9nMLvQ6m3SuxLJNgCM0foDXUUREuoScFLZm9hsz22pm7xzg/lPMrMbM3mq+fD8XuaR9Ci46kuBxg6j97jNk6uJex2kzMyNy/ChcIk18/ntexxERORw/BC4BfgkMaT62Ebjas0TS6VwyRWLFZgJD++ArLvA6johIl5CrGdvfAbMPMeZF59zE5ssPc5BJ2snMKLtrNpnN9dTd/JLXcdrF36uQ0PiBuEQKl9G5tiKSty4GznbO/QnY88fsPWCEZ4mk82UgNLY/4SMHeZ1ERKTLyElh65x7Aci/PWLkgELTBlHwmaOpv/0VUmt3eR2nXcLHDiN62hGYT/v+iUje8gP1zdf3FLZFex2TbsjCASLHDsNfUex1FBGRLqMrnWN7vJm9bWaPm9l4r8PIoZXe9BHMZ9ReM9frKO2yp6BN1zSSXLfd4zQiIu3yOHCHmYUhe84t8CPgUU9TSadJbd5NcsMOdfYXEdlPqwtbMzvVzIY3X+9vZv9jZr81s8oOyLEAGOqcmwDcAzx8kByXmtl8M5u/bdu2DnhpaS//oFKKvnMCsQeXEH95vddx2i0+fy2xF94l05B/5wuLSI93JdAfqAFKyc7UDkXn2HZLzjni89fSNG/NP+fnRUQEaNuM7X1Auvn67UAQyJBtWHFYnHO1zrn65uuPAUEz63OAsb90zk12zk2uqKg43JeWw1T07RPwDSym5ooncJn82/4HIDJ1ODhH0+trvI4iItJqzbOzfYALyDaOOg4Y6Zz7mHOuztNw0inSW2tJb6sjPH6gTqMREdlPWwrbgc659WYWAGYBlwKXA9MPN4SZVTa/QWNmU5tz7Tjc55XO5ysMUXrzTJLzNxH7/SKv47SLr7iA8NGDSa3dTqo6P88XFpGex2XXoi4GMs65rc65N5xzm73OJZ0n8U41Fg4QrOrndRQRkS6nLYVtrZn1A2YAS/fMsJKduT0oM3sAeBUYY2YbzewSM7vMzC5rHvIJ4B0zexv4GXCR08kjeaPgU0cRnDKAmmufJtOQ8DpOu4SOGoSvpICmV1fhUvk58ywiPdJCYLTXIaTzpWtipNbvIDimPxbwex1HRKTLCbRh7D3AG0AIuKL52AnA8kM90Dn3yUPc/3Pg523IIl2I+XyU3jWb7Sf8hvpbX6bkhlO9jtRm5vcROX4kqY27QN+piEj+eA54wsx+B2xgrzMvnXO/OdSDzWw2cDfZ7sq/cs7dvN/9FwM/BaqbD/3cOferjggubeMa4vhKCgiNG+B1FBGRLqnVha1z7hYz+zuQds6tbj5cDXyxU5JJXglPH0LBRUdSd+vLRC+ZRGBImdeR2iwwoBeBAb28jiEi0hYnkN23dsZ+xx1w0MLWzPzAvcDpwEbgDTP7h3Nu6X5DH3TOfa2D8ko7BQaUUfjxY2k+c0tERPbTlhlbnHPv7rluZqeSPa/n+Q5PJXmp5OaZxB5eTu21T1P+h/O9jtNuqc01JNduIzJtpD5AiEiX5pw7nCUyU4FVzrk1AGb2J+BcYP/CVjyW3tmAr7QA83elXRpFRLqWtmz387yZndB8/WrgT8Afzey6zgon+SUwtIziq44n9sfFJF7b4HWcdkvvrCe57H1S2ttWRPKAmfUys8+Z2bXN/7Z26clAssuX99jYfGx/55vZIjP7q5kNPuzA0iYunaHxycXEXnr30INFRHqwtnz1dyTwWvP1LwGnkt1a4LIDPkJ6nKJrTsRXWcTuK57I283jQ2MH4CsvpGneGlwy5XUcEZEDMrPjgdVk34uPBr4MrG4+3hEeBYY5544GngL+5wA5tMd8J0mu2YqLJQmpE7KIyEG1pbD1Ac7MRgLmnFvqnNsA6KREaeErClNy00dIzqsm9sBir+O0i/mMyPRRuMYE8YXrvY4jInIwdwFfcc5Nd8590jl3Atmt+H7WisdWA3vPwA7in02iAHDO7XDOxZtv/go49sOeSHvMdw7nHIl3qvH1KsTfP/96V4iI5FJbCtuXyHYuvg34O0Bzkav1mrKP6OcmEDymP7VXzyXTmJ/b/wQqSgiOriSxtJr07gav44iIHMho4M/7HfsrMKoVj30DqDKz4WYWAi4C/rH3ADPrv9fNc4Blh5FV2ihdvYvM7kZCRw5UzwcRkUNoS2F7MbAbWAT8oPnYWLLbBIi0MJ+P0jtnkd5YS/3tr3odp90ik4cROW4kvpKo11FERA5kJdmCdG8XkF2efFDOuRTwNWAO2YL1z865JWb2QzM7p3nYN8xsSfM+898g+1lAciS5YScWDREcrllwEZFDsXw9DxJg8uTJbv78+V7HkAPYccGfiT+2kn7vfh3/wBKv4xwW55y+LRfpAczsTefcZK9ztJaZTQf+H/AusA4YBlQBZzvnXvEik96bO45zDteQwFcU9jqKiIhnWvve3JauyEEzu8HM1phZU/O/NzQvXxL5gNJbT8elMtRc97TXUQ5LauNOGv7+JpmmpNdRRET20Vy8jiR7qtCbwD3AKK+KWuk4Lp3BzFTUioi0UluWIt8KzCTbeXFC87+nAbd0Qi7pBgLDe1F05XHE7n+bxBvVh35AF2WFYTK1MeJvrvU6iojIPsxsIIBz7vfOuVudc78n2+hxgMfR5DBkGuLUPziPpLadExFptbYUthcA5zjnnnTOrXDOPQl8DLiwc6JJd1B83Un4+hZSc2X+bv/j71VIaPxAku9uJrW11us4IiJ7e5hsN+O9DaK5yaPkp8SyTbhECn95kddRRETyRlsK2wOdYKgTD+WAfCURSn5yGomXNxD7yxKv47RbeOJQLBqi6dVVuEx+Fugi0i2Nds7ts7da8+2xHuWRw+SSaRIrNhMY2gdfccTrOCIieaMthe1fgEfNbJaZjTOz2WS/Kf5L50ST7iL6hUkEJ/Sj9jtP4WL5eZ6qBf1Epo0ks7OB1IYdXscREdljm5nts7VP8239ocpTiZWbIZEiNH6g11FERPJKWwrb7wBzgXv5Z4OKZ4Fvd0Iu6UbM76P0ztmk19VQf2f+bv8TGNqb6FkTCA7t43UUEZE9fgM8ZGZnm9kRZvZR4CHgVx7nknZwzpFYugl/3xICffN7NwERkVwLHOxOMzttv0PPNV8M2LMe80TgmY4OJt1L+NThRM4bS91NLxH9wiT8/Yu9jtRmZtbyQSO1rRZ/aRQLHfRXSESks90MJIHbgMHAerJF7Z1ehpL2MTOip42DtE55ERFpq0N9Kv/1AY7v+Yu7p8Ad0WGJpNsq/enpbDniXmq/9wy9fn2u13HazcWTNM55B395IdEzjsQCfq8jiUjPNQP4q3Pup2bWn+xOBUcCfYHNniaTdlHDKBGR9jnoUmTn3PADXEY0X4Y751TUSqsERvWm6JvH0fjbhSQWbPI6TrtZOEjB9FGkt9TS+MwyXDrjdSQR6bnuA9LN128n+4V1BvilZ4mkXVJbaml8fjmZxoTXUURE8lJbzrEVOWzF3zsZX+8oNd+ak7fb/wAER/QlckIV6epdxJ5bjsuouBURTwx0zq03swAwG7gUuByY7m0saavEOxtJbdyFBbUKSESkPVTYSk75Sv9/e3ceXkV5/338/T1bkpM9ZAHCLgFZXBAKLqi4o1VsrVUorq0Fqj6tXZ7W2p/92cXa+nTRWlfUWltXrLvWfakLoigomyA7BiQhhISsZ7ufPxJiUEQCSSYn+byu61zkzAwzn8k1yX2+ue+5J5Ws3xxD5NV1NDyyzOs4+yQ0rDcpE4YQW19BdMVmr+OISM9UbWZFNA1JXuKcq2leHvQwk7RRorqe2PoKQvv3UWErIrKXNPONdLrwRYdQc+M7VJ7/KInyOsIzxmKWnI9DThlZjC8rjUBxrtdRRKRnugF4BwgBlzUvOwL40LNE0maNS0rBZ4RG9PU6iohI0lKPrXQ6C/jp9fR0QhOK2TbrSSpO+iex9du8jrXXgv3yMDMSNQ00frAhqYdYi0hycc79ATgeOMI5d3/z4lLgIu9SSVskGqJEP9pMcL9CfOGQ13FERJKWClvxRKB/Nr2eP4+cm79K5M0NlI2+idrb303qojC6cjON766lceF6r6OISA/inFvhnFv1mfeLvMwkbRMa0YfQqGKvY4iIJDUVtuIZMyN91lcoXHQxwXF92fbdJ6g4+V/EP67yOtpeCR00gGBJEZGF62lctMHrOCIikgR8qUFSvzIEf26611FERJKaClvxXGBwLvkvnEf2304h8tp6No+6idq/L0i63lszI/XwEgKDC2icv5bIsuR9pJGIiHS8WGklsdLKpGvvRES6IhW20iWYz0fGJeMp/OB7BA/uzbZvP0bFqfcSL632OlqbmM9IO2oYgf55RD7arMcAiYjILjnnaHh7NQ3vrPE6iohIt6DCVrqUwH555L98PtnXTyby8ho2j7qR2n8sTKq/ZpvPR9qkEaRPPgDz6UdMREQ+L15aSWJbHaHRxUn7ZAARka5En7qlyzGfj4zvH9rUe3tAEdsueJStU+4jvjF5em8t4MNCAVwsTt2LS4lu2Op1JBER6UIaF5di4RDBwQVeRxER6RZU2EqXFRjai/xXLiD7LyfR8MJqNo++ibp7Pkiq3lsSjkRtI/UvLyW2MXkfaSQiIu0nXlFDfNM2QiP6Yn59FBMRaQ/6bSpdmvl9ZFx2GIXvzyI4ooDKcx5m69fvJ/7Jdq+j7RELBQifOBpfZhp1Ly4hVpY8vc4iItIxEnURfFlphIb39jqKiEi3ocJWkkJwWD75/72QrD+eSMMzK9k86ibq7luUFL23vtQg4ZMOwJcWou75xcFLWhcAACAASURBVMQraryOJCIiHgr2zyP9jLFYStDrKCIi3YYKW0ka5veR+ePDKVw4i0BJHpXf+jdbz3yQeFnXLxR94RDhyQfgy0zzOoqIiHgoXlGDSyQ0YZSISDtTYStJJ7h/AQWvf5usPxxPw5MrKBt1E3UPLvY61pfyZaSSftrB+HtlAOAaYx4nEhGRzuSiMWqfWUTD3FVeRxER6XZU2EpSsoCfzJ9OpHDBTPyDc6g8+yG2nvUg8fJar6Pt1o6/0Dd+sIGax98jUdPocSIREekskRWbIRIjOKzI6ygiIt2OCltJasGRhRS8+R2yfncc9Y9+SNmoG6n/91KvY32pQN8cXGOMumcXkaiPeB1HREQ6mEs4IktL8RdmESjI8jqOiEi3o8JWkp4F/GT+/EgK35uJf0A2W898kK1T5xDf0nV7b/35mYRPGEWirpG6ZxfhGqNeRxIRkQ4UW7cFV9NIaHSx11FERLolFbbSbQRHF1Ew9yIyf3MM9Q8vo2zUTdQ/sszrWF8oUJRN+LiRJKrqqXt+CS7R9Wd4FhGRvRPbsBVfZiqB/r28jiIi0i2psJVuxYJ+sv7naArnz8DfN5OtZzzA1un/Jl5R53W0XQr0zSXtmBGERvTFfJohU0Sku0o9chjhkw/U73oRkQ6iwla6peCBvSl4+7tk/moS9Q8uoWz0TdQ//qHXsXYpOKAXwf0KAYiVV+PiCY8TiYhIe3Lxpsf7+NJTvI4iItJtqbCVbsuCfrJ+OYmCd76LrzCdraffz9bzHiZRWe91tF1K1DRS958PqH/lQw1LFhHpJuJV9Wy/fx6x0kqvo4iIdGsqbKXbCx3ch8J3vkvmL4+m/t5FbB51I/VPLvc61uf4MlJIGTeY2PoKGl5bjnMqbkVEkl1kaSnE4vhy072OIiLSramwlR7BQgGyfnUMBW9/F19+mK2n3UflBY+Q2Na1em9TRhaTMnYQ0dXlNLy5UsWtiEgSSzREiX60meB+hfjCIa/jiIh0aypspUcJHdKXwndmkPmLI6n71wdsHn0TDf/5yOtYO0k5sD+hA/sTXfEJsbVbvI4jIiJ7Kbp8E8QThEbpET8iIh1Nha30OJYSIOu3x1Hw1kX4slOpOOUeKi96jERVg9fRWqQcMpC0Y0cQGJTvdRQREdkLLpEgsmwj/uJc/BqGLCLS4VTYSo8VGldM4Xszybh8InV/X0jZ6JtoeG6l17EAMDOCA/MxMxLV9UQ+3OR1JBERaQPz+QgfP4rUcYO8jiIi0iOosJUezVICZF9zPAVvfgfLCFFx0r+onPE4iequ03sbWbqRhrkriSzb6HUUERFpA39+Jv68DK9jiIj0CCpsRYDQhH4ULphJxv89nLo7FlB2wM00vLDK61gApIwfTKB/Hg1vrSLy0Wav44iIyJeIlVZS/9oKEg1Rr6OIiPQYKmxFmllqkOxrTyT/9W9jqQEqTvgn2773JIntjd7m8vlImzQCf98cGt5YQXRtuad5RKT7MLPJZrbczFaa2eW72e4bZubMbFxn5ktWjYs/JraxEgv6vY4iItJjqLAV+YyUw/pTuHAWGT8+jNpb51N24M00vrzG00wW8BE+diT+giwiSzfqMUAiss/MzA/cCJwMjASmmdnIXWyXCfwAmNe5CZNTfGsN8Y3bCI3oi/n1MUtEpLPoN67ILlhakOw/nkT+a9+GgI8tx/6DbZc+RaLGu95bC/oJnzCK8AmjMDPPcohItzEeWOmcW+2ciwD3A6fvYrvfAH8Aus7kA11YZHEpBHyEhvf2OoqISI+iwlZkN1KOGEDh+7NIv+xQam96p6n39tW1nuWxUAALBnDROHUvLiVWVu1ZFhFJesXAhlbvP25e1sLMDgH6O+ee6sxgySpR20h0dTmhkt5YStDrOCIiPYoKW5Ev4QuHyPnLZPJfvRB8xpZJd7HtB/8hURvxLJOLxUlsq6Pu+cXEK2o8yyEi3ZeZ+YA/Az/eg21nmNl8M5tfXt6D5wEwI7R/H0Kjir98WxERaVcqbEX2UMqRAyl8/3uk/5/x1P51HmUH3Uzja+s8yeJLCxE+aTQWDFD37CLi22o9ySEiSa0U6N/qfb/mZTtkAqOBV8xsLXAo8PiuJpByzt3mnBvnnBtXUFDQgZG7Nl84ROqh++HLTPU6iohIj6PCVqQNfOkhcv56Cvkvnw8Jx5aj/862Hz5Doq7ze299GamETzoAfEbdM4tJVNd3egYRSWrvACVmNtjMQsBU4PEdK51zVc65fOfcIOfcIOAtYIpzbr43cbu26IYKYpu2aXI/ERGPqLAV2QspkwZT+MH3SL/4K9Re9xZlB99C45vrOz2HPzuN8EkHYCkBXDzR6ccXkeTlnIsBlwLPAsuAB51zS8zs12Y2xdt0ycUlHA1vrabxvXWa3E9ExCMqbEX2ki8jhZy/fZX8F8+DSJwtE++k6ifP4uqjnZrDn5tO+tcOwZ+bjnMOF4116vFFJHk55552zg1zzu3nnLu6edkvnXOP72LbSeqt3bXYui24mgZCo3VvrYiIV1TYiuyjlGOHULjoYtJnjqPmT3MpG3MLkbc2fPl/bEc7egga31tH7VPv4xo7t7gWEempnHNElpRimakE+vfyOo6ISI+lwlakHfgyU8i5+VR6PXcuri5K+RF3UvWz53ENnVtgBvpkk6iqp/a5xbiIem5FRDpavKyaePl2UkYVYz4NQxYR8YoKW5F2lHrCfhQuvpjwt8dQc+0blB1yK5G3P+604wf65pJ2zAgSFbXUvbAEF4t32rFFRHoiVx/Bl51GcGiR11FERHo0FbYi7cyXlUru7Cn0euYc3PYI5YfdQdUVL+AaO6cHNTigF2lHDSe+uZr6l5dphk4RkQ4UHFRA+tfHYkG/11FERHq0TilszexOMyszs8VfsN7M7K9mttLMPjCzQzojl0hHSj1paFPv7QUHU3PN65SNvZXI/NIv/4/tIDikgNSJJQRLemuGThGRDhLfUoNLOP2eFRHpAjqrx/YuYPJu1p8MlDS/ZgA3d0ImkQ7ny04l947T6fXUt0hUNlB+6O1UX/lSp9z/GirpTXBQPgDx8u24hHpuRUTaS6IhSu3T79M4f43XUUREhE4qbJ1z/wW27maT04G7XZO3gBwz69MZ2UQ6Q+opwyhafDHhcw5k+2//S9m424i8t7FTjh3fVkftUwtpmLtSw5JFRNpJ9MNNEE8QLNG9tSIiXUFXuce2GGj9fJSPm5eJdBu+3DRy7/o6eU9MI7GljvIJt1P9vy93eO+tPydM6ID+RFd8QuPbq1XciojsIxdLEPlwI/7iXPy56V7HERERuk5hu8fMbIaZzTez+eXl5V7HEWmztFOHU7T4YtKmjmb7r1+lfPxsou9/0qHHTDlkIKERfYks3UjjgnUdeiwRke4uuroMVx8lZbT+Bi8i0lV0lcK2FOjf6n2/5mWf45y7zTk3zjk3rqCgoFPCibQ3X16YvH+eQd6jU4l/UkPZuNuo/vUruGjHPJ7HzEiZMIRgSRGR9zcQ21jZIccREekJYhu24stLx98nx+soIiLSrKsUto8D5zXPjnwoUOWc2+R1KJGOlnb6/hQuuYS0s0ax/X9foXzCbKKLNnfIscyM1MNLSDt6uD6MiYjsg7RjRxA+YZRmQxYR6UI663E/9wFzgeFm9rGZfcfMZpnZrOZNngZWAyuB2cDFnZFLpCvw9wqTd883yHv4bOKl2ykbeyvbr/4vLtb+vbfmM4JDCjEz4tvqiK7WcH4RkbZw8QRmhi+c4nUUERFpJdAZB3HOTfuS9Q64pDOyiHRVaV8fQejIAVRd+jTV//MS9Y8sI/eurxEc3TEzbjYuXE9sbTn4IDhIw/pFRL5MvKKGumcXkXbsSAK9s72OIyIirXSVocgiAvjz08m7/5vkzfkm8XVVlI29je3XvNYhvbdpR5TgL8ii/pXlRDfs7mlcIiICEFlSiosn8OeGvY4iIiKfocJWpAtKO3MUhUsuJvW0YVRf8SLlh99BdGlZux7Dgn7CJ4zCl5dO/ctLiW3a1q77FxHpThK1jURXlxMa1htLCXodR0REPkOFrUgX5S/MIG/OWeTefybx1ZWUHXIr2699HRdPtNsxLBQgfOJofJlpRBbvciJyEREBIss2Ao7QSD3iR0SkK1JhK9KFmRnhs0dTuOQSUk8pofpnL1A+8U6iH7bfpE++1CDhyQeQdsz+7bZPEZHuxMXiRJZ/QmBgPr7MVK/jiIjILqiwFUkC/qIM8v59Nrn3nEFs+RbKDr6F7X96s916b31pISzgx0Vi1L24lPi22nbZr4hId2CBpls3UsYM8DqKiIh8ARW2IknCzAh/60CKllxC6klDqf7Jc2w56u9EV2xpt2O4hijx8mrqnllMorq+3fYrIpLsAoVZ+HPSvY4hIiJfQIWtSJLx98kk79Gp5P7z60SXllN20C3UXDcXl9j33ltfVhrhkw6ARILaZxeRqGlsh8QiIskrum4L9W98hIvEvI4iIiK7ocJWJAmZGeFzDqJoycWkHj+Eqh8+y5ZJdxFbWbHP+/bnphM+cTSuMUbds4tI1EfaIbGISPJxzhFZ9DHxTdsg4Pc6joiI7IYKW5Ek5u+bRd7j08i562tEP9hM2YE3U3PDvH3uvfXnZxI+YRQYuEj7P0NXRCQZxMuqiZdvJzSqGPOZ13FERGQ3VNiKJDkzI/38gylacgmhSYOo+v5/2HLsP4it3rpP+w0UZZP+tbH4s9NwzuFiKnBFpGeJLC7FUgIES4q8jiIiIl9Cha1IN+EvzqLXU9PJuWMK0QWfNPXe3vj2PvXe7uihaJy3mrrnFqu4FZEeI15VT2x9BcHhfTANQxYR6fJU2Ip0I2ZG+rcPoXDxxYSOGEDVpU+z5fi7ia2t3Kf9+guziJdVU/fi0nZ7xJCISFdmAR/B/fsQGtHX6ygiIrIHVNiKdEOB/tn0euYccmafRnT+RsoOuJnaW97BObdX+wsOKSD18BLiG7dR/8qH7TIDs4hIV+ZLTyHtsKH4wiGvo4iIyB5QYSvSTZkZ6ReNbeq9PbQf2773FBUn/pP4xuq92l9oWG9SJwwhtr6ChjdWtnNaEZGuI7p2C7HNVV7HEBGRNlBhK9LNBQbk0Ou5c8m55VQib26g7OBbaHh27wrT0MhiUsYNJjAov51Tioh0DS6eoOGtlTS+v8HrKCIi0gYqbEV6ADMjfeY4CubPwFeUQcXkf1F1xQt7NRlUygH9CPbPAyBeUbPXw5tFRLqi6KoyXH2UlNHFXkcREZE2UGEr0oMERxRQMO8iwhcdQs01r7PlmH8Q/3jvhtvFyqqpfWIBjQvXt3NKERFvOOeILCnFl5uOv0+O13FERKQNVNiK9DC+cIjc2VPI/dcZRBdsahqa/J+P2rwff0EmwaFFRBaup3GRhuyJSPKLl1aS2FZHaHQxZuZ1HBERaQMVtiI9VHj6gRS8OxNf30wqTrmHqsufx0X3fGiymZF6eAmBwfk0zl9LZNnGDkwrItLxEo0xfHnpBAcXeB1FRETaSIWtSA8WHJ5P4bzvEp4xlpo/vMGWSXcR27DnQ5PNZ6QdNZxA/zwa5q3CNcYA9ureXRERr4X2KyR9yhjMr49HIiLJRr+5RXo4SwuSe+tp5N77DaIfbG4amvzUij3//z4faZNGEDpoAJYSAKD+lQ/ZPudt6l9bQeSjzSS2N3RUfBGRdhHfsh2XcBqCLCKSpFTYiggA4WkHUPDuDAL9s6g49V6qfvrcHg9NtoCP1DEDW94HBvTCn5dBbEMFDa+voOahd6h7cWnL+kRdRLMpi0iXkahtpPbJ92l8X5PhiYgkq4DXAUSk6wgOy6fgrYuo+uGz1Py/N4m8vp7c+88kMKBts4OGhvUmNKw3zjkSlXXEP6lq6c11sTg1c97GUoME+uTg752NvygbX1aqekpExBORpRsBR2hokddRRERkL6nHVkR2YqlBcm4+ldz7zyS6uIyyg2+h/onle7cvM/x56YRG9iW4X2HTQgepE4bgL8witrGShjc+ovbh+c0fLMFF48Sr6tSjKyKdwkVjRFZsIjAwH19mqtdxRERkL6nHVkR2KXz2aEJj+7D1rDlsnXIfGT86jKxrjsNC+/Zrw4J+Qvv3JbR/36Ye3ap64p9U4e+TDUBs4zbqX1qKpQXxF2UT6JONv3c2vuywenRFpN1FVmyGSJzQ6GKvo4iIyD5Qj62IfKHA0F4UvPkd0i/+CjV/nkv5UX8ntm5bu+3fzPDnhAnt3wd/dhhoej5u6uFD8ffJIV5WTcPcVdQ+8h6JqnoA4tvqiFfWqkdXpB2Y2WQzW25mK83s8l2sn2Vmi8xsoZm9bmYjvcjZkWIbKvAXZhEoyPI6ioiI7AP12IrIbllqkJwbv0rKpEFUXvQ4ZQffQu5dXyPt9P075Hi+cIjQ8D6EhvfBOYfb3kBsczW+7DQAIos/JvrRZiwl0HR/bu9sAr1z8Oeld0geke7KzPzAjcAJwMfAO2b2uHNuaavN7nXO3dK8/RTgz8DkTg/bgcInHoBriHodQ0RE9pF6bEVkj6R9cxSF780kMCSXrV+7n20/fAYXiXXoMc0MX1YaoZKilmHIKQcPIHXiMAL984hX1NA4bzX1L336OTy6voJ4RQ0uoR5dkS8xHljpnFvtnIsA9wOnt97AOVfd6m060G1+sJxzuHgC8xm+cMjrOCIiso/UYysieyywXx4Fb36Hqv/7HLXXvUXkjfXkPfBNAoNzOy2DLyOVUEkqlDTNXpqoaSBRFwGaPqg2vL4C1xiDUIBAUVZTj26/XPw56tEV+YxiYEOr9x8DEz67kZldAvwICAHHdk60jhcvq6b+pWWETxiFPz/T6zgiIrKP1GMrIm1iKQFy/noKeQ+dRWxFBWVjbqH+kWWe5fFlpBIobLo3zsxIP/0Q0o4aTnBQPomqehrfWUN0xWYAXDxB46KPiZdvV4+uyB5yzt3onNsP+BnwP7vaxsxmmNl8M5tfXl7euQH3UmRxKTiHLyfsdRQREWkH6rEVkb2S9o2RBMf0ZuvZD7H1jAdI//4Esq89oeV5tV7xpafg26+w5fFCibrGlsGTicpaGuevoREg6G+aMKZ3NsEhBfgy9JgP6XFKgf6t3vdrXvZF7gdu3tUK59xtwG0A48aN6/J/NYpX1RNbX0HooP5YwO91HBERaQfqsRWRvRYYkkfB698m/QcTqP3rPMqPuIPY6q1ex9qJL5yCLz0FAH9+JhlnTyDt6P0J7leIq22k8d21JGoaAIiXb6fx/fXENlfj4gkvY4t0hneAEjMbbGYhYCrweOsNzKyk1duvAh91Yr4OE1laCj4jtH9fr6OIiEg7UY+tiOwTSwmQc93JTbMmX/gYZWNuJffO00n7Rtd8KogvHMI3pIDgkAIAEvWRlmfzxjZX0fjeOmAd+H0tPbqh0cXq1ZFuxzkXM7NLgWcBP3Cnc26Jmf0amO+cexy41MyOB6JAJXC+d4nbh2uMEv1oM8H9CjVplIhIN6LCVkTaRdrXRhA8uHlo8pkPkn7peLL/eKLnQ5O/jC/t0w+2KaP7ERxaRPyTKuKfVBH7pIrI0lJCBzaN1mxcWgqNMfx9svHnZ2EBDXqR5Oacexp4+jPLftnq6x90eqiOFgoQPnG0iloRkW6ma3/iFJGkEhiUS8FrF1J1+QvU/uUtIm9uIO/BbxLYL8/raHvMlxrENyif4KB8AFwsjvmaHjUU31xNbO0WWAj4DX9BFsGBvQiNLPYwsYi0hZkR6J3tdQwREWln6m4QkXZloQA5f55M3mNTia2ppOyQW6mfs8TrWHut9RDk8DEjyPzWoaQdN5LQ/n1x0Tjxilqg6VFDdS8upeHdtcQ2VuKica8ii8gXiK4qo37uSlxMP58iIt2NemxFpEOkTdmf4IJZbD17DlvPmkP6xWvJ/tOJWGrQ62j7xFKCBAf0IjigF9BU0AIQiZOoixDbUEHkgw1ghr8gk9AB/QgO6IVLJCDusKDu1RXxgnOOxg82gM/Ar7/ri4h0NypsRaTDBAbmUPDfC6m+4kVq/jT306HJJb28jtZuzJqGKVtKgIzTDsZFY01Dlpvv06W58I1vqaHuqfextBC+rFR8WWn4stIIDi7Al6lHDYl0tFhpJYltdaQeOazl51ZERLoPFbYi0qEsFCD7jycRmjSIyvMfpWzsreTMnkL47NFeR+sQFgwQ6JdHoN/O9xX7wiFSDhlIorqBxPZ6Yh9vxdVH8Rdm4stMJbq2nIZ5q1sKXl9mU/EbKM7BgvpVLbKvIotLsXCI4OACr6OIiEgH0KclEekUaacOJ7hgJpVTH6Jy6kM0vryGnL9MxtKSe2jynvJlpJJy0ICdlrlIrGVIpKWFCPTNIVHdQGx9Ba4hCkDGmeOwYIDIik+IriprLnw/7fH1ZYdbJrcSkV2LV9QQ37SNlLGDMA1DFhHpllTYikinCQzIIf/VC6n+n5eoufYNom99TO6D3yQ4LN/raJ7Y8fxcgEBRNoGiT2dqdZEYiep6LL15mLIZJNxORS9A5rlHgM+IfLiReEXtzoVvZpoeSSRC089acHhvQsN7ex1FREQ6iApbEelUFvST/YcTSDl6IJXnPUL52NvIue00wtMO8Dpal2KhAP78zJb3oZIiQiVFALjGGInt9SRqG1sK10R1A7F1W3CNsU/3kRYkc+qhAESWb8JF4s29vKn4MlT0Ss/hy0wl7fASr2OIiEgHUmErIp5IPWUYBQtmUTntISq/9e+mocnXn9xjhibvC0sJ4E/J3KnwTR0/hNTxQ3CN0ab7eKvrcfFEy/roqjLim6t32k+gfx7h40e1rCfob7m/V8M1pbuIri7Dl5mGvyDzyzcWEZGkpcJWRDwT6J9N/ssXUP3Ll6n5/etE3vqYvAe/SXB/Te6ytywliL8g+LkP8emnHLRT0RuvrsfX6tFLDfNWfdrba2DpKYSGFpEyZiDQNKOsZaTgy1DRK8nDRWPUz11JoG8u4WNGeB1HREQ6kApbEfGUBf1kX3N809Dkcx+hfNxt5Nx6GuHpB3odrdtpXfR+tl88/YxxJKrrm17bm4pfS2naykXj1D23uHknYOmp+LJSCQ3rTXBwAS7hSGyvb+rp9XVO0euca3lki4vG9Xxg2aXIis0QiRMaXex1FBER6WAqbEWkS0idXELhwllsnfYQlec8TOPLa8j+68n4wiGvo/UIvtRgUw9uYdbnV/qN8FcP+rTw3THUORoHILG9ntqH320qejNSWyawCg0twp+fiUs4wO1x0euiMRJ1kabJr3xGbGMlsfVbSdRHcLWNTf/WR8k85zDM5yNR24g/J9yO3w3pDlzCEVlair8wi0DBLq5rERHpVlTYikiX4S/OIv+l89l+1Sts/91rROaVNg1NHqGhyV4yn49AYdaui16aiuLUicOaJrRqLnyjZdUE+uTgz88k/sk26p5b3DSMOSsNf1YalpVKaL8iLCVAdEMFkSWluLoIiboINBfMGd/8CpaRSnxLDZGVm/GFQ1g4RKAoGwuHIO7AB76MlM78dkiSiK3bgqtpJDR+iNdRRESkE6iwFZEuxQJ+sn57HKGjBlJ5zsNNQ5Nv/irh8w72Opp8AUsJtszYvINzDlzz+vQUQgf0b+nxjWyuhlgcf6+MpkccJYC4w5ebTqA4FwuH8IVTWh6HFDqgHykH9v/i4wc0DFk+z0Vi+PIzCPTv5XUUERHpBCpsRaRLSj1xaNPQ5G/9m8rzH6XxlbVk33AKvnQNTU4GZgZNt8Dizw7jHzuoZZ1zDtcQbbkvNjiwF8GBX1x87LiXVqQtQsP7EBzWW9ePiEgPoaktRaTL8vfNIv+F88i88ijq7lpI+fjZRJeWeR1L9pGZ4UsLqadVOkx8y/adJhgTEZHuT4WtiHRpFvCT9etj6fXcuSS21FE+7jZq71rgdSwR6aLiVfXUPrGQyJJSr6OIiEgnUmErIkkh9fj9KFw4i+Ch/dh24WNsPf8RErURr2OJSBcTWVoKPiM4pNDrKCIi0olU2IpI0vD3yST/+fPI/N+jqf/n+5R/5Taiizd7HUtEuohEQ5ToR5sJ7leoR4WJiPQwKmxFJKmY30fWVcfQ6/nzSGytp3z8bGrveK9pFl4R6dGiyzdBPEFoVLHXUUREpJOpsBWRpJR63BAKF84idHh/tl30OJXnPUKiptHrWCLiEecc0fVb8Rfn4s9N9zqOiIh0MhW2IpK0/L0z6fXsuWT++hjq711E+bjbiC7S0GSRnsjMSP/qgaRNHOZ1FBER8YAKWxFJaub3kXXl0eS/cB6JqkbKxs+mdva7Gpos0oM453DxBObz6d5aEZEeSoWtiHQLKccMpnDhLFKOHMC2GU9Qec7DJLZraLJITxArraRmztvEK2u9jiIiIh7ptMLWzCab2XIzW2lml+9i/QVmVm5mC5tfF3VWNhHpHvxFGfR65hyyfnss9fcvbhqa/P4nXscSkQ4WWVwKZviy0ryOIiIiHumUwtbM/MCNwMnASGCamY3cxaYPOOcObn7d3hnZRKR7MZ+PzF8cRf5L55OoiVA2YTa1t87X0GSRbipeUUN80zZCI/pifg1EExHpqTqrBRgPrHTOrXbORYD7gdM76dgi0gOlHD2oaWjypEFsm/UkldMeIlHd4HUsEWlnkSWlEPARGt7b6ygiIuKhzipsi4ENrd5/3Lzss75hZh+Y2UNm1r9zoolId+UvSKfX09PJ+t1x1M9ZStnY24gs2OR1LBFpJ4m6CNHV5YRKemMpQa/jiIiIh7rSmJ0ngEHOuQOB54F/7GojM5thZvPNbH55eXmnBhSR5GM+H5k/P5L8Vy7A1UcpP+x2am5+R0OTRboBSwsSPnEUodG7+lu5iIj0JJ1V2JYCrXtg+zUva+Gcq3DO7ZjC9HZg7K525Jy7zTk3zjk3rqCgoEPCikj3k3LkwKahyccOpurip6g8ew6JKg1NFklmZkagby6+jFSvo4iIiMc62sIPWAAAFzRJREFUq7B9Bygxs8FmFgKmAo+33sDM+rR6OwVY1knZRKSH8Oen0+vJb5H1++Opf3gZZWNvJfLeRq9jicheiCzfRMO8VbhEwusoIiLSBXRKYeuciwGXAs/SVLA+6JxbYma/NrMpzZt938yWmNn7wPeBCzojm4j0LObzkfmzieS/eiE0xik/7A5q/jZPQ5NFkohLOBo/2EC8ogbzdaW7qkRExCuBzjqQc+5p4OnPLPtlq69/Dvy8s/KISM+WcsQAChbOovL8R6j6P/+h8ZW15N5xOr5sDWkU6epi67bgahoJjR/idRSRHqm6upqysjKi0ajXUSTJBYNBCgsLycrK2ud9dVphKyLS1fh7hen1+DRq/jyX6stfoGzBJ+Q9cCahcZqIRqSrcs7RuLgUX2Yqgf69vI4j0uNUV1ezefNmiouLSUtLw8y8jiRJyjlHfX09paVNUy/ta3Gr8Tsi0qOZz0fmT44g/7VvQzRO+eF3UPPXtzQ0WaSLipdVk9iyndCoYsynD9Qina2srIzi4mLC4bCKWtknZkY4HKa4uJiysrJ93p8KWxERIOWw/hQunEXq5KFU/eAZtn7jARLb6r2OJSKfYalBgsN6Eywp8jqKSI8UjUZJS0vzOoZ0I2lpae0yrF2FrYhIM19emLzHppH1pxNpeGIFZWNuJfL2x17HEpFW/Nlh0o4owQJ+r6OI9FjqqZX21F7XkwpbEZFWzIzMHx1OwWsXgnOUT7yTmuvmamiySBcQ+Wgz8Yoar2OIiEgXpMJWRGQXQof2p3DBLFJPKaHqh8+y9ev3k6jU0GQRryQaojTMXUlkmZ49LSIin6fCVkTkC/hy08h7ZCrZfzmJhqc/omzMLUTmaWiydB9mNtnMlpvZSjO7fBfrf2RmS83sAzN70cwGepETIPrhJognCI3SrOUisncuuOACzAwzIxgMkp+fz8SJE7n22mupra31Op7sIxW2IiK7YWZkXHYYBW98B8won3gn2//8poYmS9IzMz9wI3AyMBKYZmYjP7PZAmCcc+5A4CHg2s5N2cTFEkQ+3Ii/OBd/broXEUSkmzjyyCPZtGkT69at4+WXX2b69On87W9/45BDDmHz5s1ex/NMJBLxOsI+U2ErIrIHQl8ppnDBTFJPG0b1j59j6+n3kdha53UskX0xHljpnFvtnIsA9wOnt97AOfeyc27Hhf4W0K+TMwIQXV2Gq4+SMlq9tSKyb0KhEL1796Zv374ccMABfO9732Pu3LmUl5dz+eU7D1y54YYb2H///UlNTaWkpISrr76aWCwGwC9+8QuGDx/+uf1/73vfY+LEiV94/Oeff55JkyaRl5dHdnY2Rx99NG+//fZO29TU1HDZZZfRv39/UlJSGDRoEL/73e9a1peVlXHhhRdSVFREamoqw4cP58477wTglVdewcz4+OOdR5gFAgHuuusuANauXYuZcc8993DKKaeQnp7OlVdeiXOO7373u+y3336kpaUxZMgQrrjiChobG3fa1wsvvMCRRx5JOBxuOYdVq1bxyiuv4Pf72bBhw07b33333WRnZ3d4r7gKWxGRPeTLSSPv32eTff1kGp5ZSdmYW2mcu+HL/6NI11QMtL6AP25e9kW+A/ynQxN9AReN4y/Kwt8nx4vDi0g3V1xczPTp03n44YdJJBIAXHXVVfzxj3/kmmuuYdmyZVx//fXceuut/OpXvwLg/PPPZ8WKFcybN69lP42NjTzwwAOcd955X3ismpoaLr74YubOncubb75JSUkJkydPpqKiAgDnHKeeeiqPP/44N9xwA8uWLePuu++moKAAgPr6eo4++mjef/997rnnHpYuXcoNN9xAOBxu83n/7Gc/Y/r06SxevJhZs2bhnKOwsJB7772XZcuWcd111/H3v/99p6L6hRde4KSTTmLs2LHMnTuXefPmcd555xGNRpk0aRIlJSUtRfYOs2fP5lvf+hbp6R074ibQoXsXEelmzIyM7x9K6PD+bD1rDluO+jtZvzuOjB8fhvn0t0LpnszsHGAccPQXrJ8BzAAYMGBAux8/ZVQxoZF99YgRkS6s9j8ffG5ZcFA+oRF9cbE4dc8v+fz6oUWESopINESpf3nZ59aHhvchOKSARE0j9a8t/9z69JMPbJ/wwKhRo6iurmbLli1kZGRw7bXX8vDDDzN58mQABg8ezG9/+1u+//3v85vf/IZhw4YxYcIE7r77biZMmADAE088QX19PWedddYXHufrX//6Tu9vu+02/v3vf/PMM88wffp0XnrpJV599VXeeecdxo0bB8CQIUM46qijALj33ntZs2YNK1eupF+/fi3r98bMmTOZPn36Tsuuvvrqlq8HDRrEqlWruOmmm1oK+l/96lecfPLJXHfddS3b7b///i1fz5gxg+uvv54rr7wSn8/Hhx9+yOuvv85f//rXvcrYFvoUJiKyF0Ljiil8byappw+n+qfPUzHlPuIVGposSaUU6N/qfb/mZTsxs+OBXwBTnHONn10P4Jy7zTk3zjk3bkevQnuJl2/HOaeiVkQ61I65M8yMJUuWUF9fzze+8Q0yMjJaXjNnzqSqqory8nKgqdf2gQceIBqNAk1DbqdMmUJOzhePLlmzZg3nnnsuQ4cOJSsri6ysLKqqqli3bh0A7777Lrm5uS1F7We9++67jBw5sqWo3Rfjx4//3LLZs2czYcIEioqKyMjI4Oc//3lLth3HP/HEE79wn+effz5lZWU8++yzANx+++2MHTuWMWPG7HPeL6MeWxGRveTLSSNvzlnU3vQOVT96lvKDbyH3gTNJObz9e6xEOsA7QImZDaapoJ0KfKv1BmY2BrgVmOycK+vsgPGKGmqfXEjq4UMJDe/T2YcXkTbYXe+pBfy7Xe9LDe5+fUZKu/bO7sqSJUvIzs6mV69erF69GoA5c+YwbNiwz22bl5cHwNSpU7nssst46qmnOOKII3jmmWd49NFHd3ucU089lfz8fG688Ub69+9PKBRi4sSJ7TZ5k6959FjrSS7j8XjLEOvWPjs0eM6cOVxyySX8/ve/5+ijjyYrK4s5c+bwi1/8Yo+P36tXL84880xmz57Ncccdx913381vf/vbvTybtlGPrYjIPjAzMi4ZT8Hc70CKny1H/Z3tf3gdt4sGRKQrcc7FgEuBZ4FlwIPOuSVm9mszm9K82f8DMoA5ZrbQzB7vzIyRJaUQ8BEclN+ZhxWRHqa0tJR77rmHM844A5/Px6hRo0hNTWX16tUMHTr0cy+/3w9Abm4up512Gv/85z+57777yMvL46STTvrC41RUVLB06VIuv/xyTjrpJEaOHElqaiplZZ/+3XDs2LFUVlYyf/78Xe5j7NixLF269HOTQ+1QWFgIwMaNnz7ze+HChXv0NIf//ve/jBkzhh/96EeMHTuWkpIS1q5d+7njP/fcc7vdz8yZM3niiSe49dZbqa+vZ9q0aV967PagwlZEpB2EDulL4bszSTtjBNWXv0DFqfcS36Jn4knX5px72jk3zDm3n3Pu6uZlv3TOPd789fHOuSLn3MHNrym732P7SdQ2El1dTmhYbywl2FmHFZFuLhKJ8Mknn7Bx40YWLVrEzTffzGGHHUZhYSHXXHMNABkZGVxxxRVcccUV3HjjjSxfvpwlS5Zw//3387Of/Wyn/Z133nk8+eST3HLLLUyfPr2l6N2V3NxcCgoKmD17NitWrGDu3LlMmzaNtLS0lm2OPfZYjjzySM4++2wee+wx1qxZwxtvvMHtt98OwLRp0xg4cCBTpkzhhRdeYM2aNbz44os88MADAAwdOpSBAwdy1VVXtdzf+sMf/nCPbucYPnw4ixYt4rHHHmPVqlVcf/31PPzwwzttc+WVV/Kf//yHyy67jA8++IDly5dz1113sXz5p/dAT5w4keHDh/OTn/yEqVOnkpmZ+aXHbg8qbEVE2okvO5XcB75J9k1fpfGlNZQdfAuNr6/78v8oIp8TWbYRcIRG6hE/ItJ+XnvtNfr06cOAAQOYNGkS99xzD5deeinvvfceRUVFLdtdeeWV/PnPf2b27NkcdNBBTJw4kb/85S8MGjRop/2dfPLJZGdns2zZst3OhgxNw4TnzJnDqlWrOPDAA7ngggu47LLL6NPn01stzIynnnqKU045hVmzZjF8+HDOOecctmzZAkA4HObVV19l9OjRTJ06lREjRnDJJZdQX18PND3W54EHHqCsrIwxY8ZwySWXcPXVV7cMUd6dmTNncu6553LhhRcyZswY5s2bx1VXXbXTNieeeCJPP/008+bNY8KECYwfP55//OMfBIM7/wHyu9/9LpFIhBkzZnzpcduL7Um3dFc1btw490Xd9CIiXoos2MTWs+YQX1NJ1rUnkPmjw72OJHvAzN51zu16xg7ZI+3RNjvnqH30PXw5YcLHjGinZCLSHpYtW8aIEfq5lN376U9/yvPPP8+CBQv2aPvdXVd72jZr8igRkQ4QGtOHwndnsG3GE/iyU72OI5JUzIz0KWNwkZjXUUREpA2qqqpYsWIFt912W6c84qc1FbYiIh3El5VK7n1neh1DJCmZ34elhbyOISIibXD66aczb948pk6dyjnnnNOpx1ZhKyLSgfTsTREREekpXnnlFc+OrcmjREREREREJKmpsBUREREREZGkpsJWRERERET2WDI/VUW6nva6nlTYioiIiIjIHgkGgy3PTBVpD/X19Z97Du7eUGErIiIiIiJ7pLCwkNLSUurq6tRzK/vEOUddXR2lpaUUFhbu8/40K7KIiIiIiOyRrKwsADZu3Eg0GvU4jSS7YDBIUVFRy3W1L1TYioiIiIjIHsvKymqXQkSkPWkosoiIiIiIiCQ1FbYiIiIiIiKS1FTYioiIiIiISFJTYSsiIiIiIiJJTYWtiIiIiIiIJDVL5udPmVk5sK6ddpcPbGmnfXWmZM0Nyu6FZM0Nyu6VZM2+t7kHOucK2jtMT9LObXNnS9brva10nt1HTzhH0Hl2J3tzjnvUNid1YduezGy+c26c1znaKllzg7J7IVlzg7J7JVmzJ2tu8VZPuW50nt1HTzhH0Hl2Jx15jhqKLCIiIiIiIklNha2IiIiIiIgkNRW2n7rN6wB7KVlzg7J7IVlzg7J7JVmzJ2tu8VZPuW50nt1HTzhH0Hl2Jx12jrrHVkRERERERJKaemxFREREREQkqfX4wtbMJpvZcjNbaWaXe51nd8zsTjMrM7PFrZblmdnzZvZR87+5XmbcFTPrb2Yvm9lSM1tiZj9oXp4M2VPN7G0ze785+6+alw82s3nN180DZhbyOusXMTO/mS0wsyeb3ydFdjNba2aLzGyhmc1vXpYM10yOmT1kZh+a2TIzOyxJcg9v/l7veFWb2WXJkB3AzH7Y/DO62Mzua/7ZTYprXbyRzG1TW3SHdmxPJWt71xbJ2ja2RbK2o22R7G1uW3Rm+9yjC1sz8wM3AicDI4FpZjbS21S7dRcw+TPLLgdedM6VAC82v+9qYsCPnXMjgUOBS5q/z8mQvRE41jl3EHAwMNnMDgX+APzFOTcUqAS+42HGL/MDYFmr98mU/Rjn3MGtpoVPhmvmeuAZ59z+wEE0fe+7fG7n3PLm7/XBwFigDniEJMhuZsXA94FxzrnRgB+YSnJd69L5krltaovu0I7tqWRu79oiGdvGtkjKdrQtkrnNbYtOb5+dcz32BRwGPNvq/c+Bn3ud60syDwIWt3q/HOjT/HUfYLnXGffgHB4DTki27EAYeA+YQNODpQO7uo660gvoR9MvxmOBJwFLouxrgfzPLOvS1wyQDayhef6CZMm9i/M4EXgjWbIDxcAGIA8INF/rJyXLta5X13gla9vUxnNMunasDeeWtO1dG88z6drGNp5ft2hH23jOSdXmtvHcOrV97tE9tnz6zd7h4+ZlyaTIObep+etPgCIvw3wZMxsEjAHmkSTZm4c2LQTKgOeBVcA251yseZOufN1cB/wUSDS/70XyZHfAc2b2rpnNaF7W1a+ZwUA58Pfm4XC3m1k6XT/3Z00F7mv+ustnd86VAn8E1gObgCrgXZLnWhePJWPb1BZJ3o7tqWRu79oiGdvGtugu7WhbJFWb2xad3T739MK2W3FNf/bostNcm1kG8G/gMudcdet1XTm7cy7umoaK9APGA/t7HGmPmNmpQJlz7l2vs+ylic65Q2i6VeASMzuq9coues0EgEOAm51zY4BaPjOMqIvmbtF8n8sUYM5n13XV7M33IJ1O0weivkA6n79tQ2SXkrVtaotkbcf2VDdo79oiGdvGtkj6drQtkrHNbYvObp97emFbCvRv9b5f87JkstnM+gA0/1vmcZ5dMrMgTR8c7nHOPdy8OCmy7+Cc2wa8TNOQiRwzCzSv6qrXzRHAFDNbC9xP0/Cs60mO7Dv+yodzroym+07G0/WvmY+Bj51z85rfP0RTA93Vc7d2MvCec25z8/tkyH48sMY5V+6ciwIP03T9J8W1Lt7pDm1TWyRhO7ankrq9a4skbRvboju0o22RjG1uW3Rq+9zTC9t3gJLmmblCNA0FeNzjTG31OHB+89fn03SPUJdiZgbcASxzzv251apkyF5gZjnNX6fRdP/VMpo+GJzZvFmXzO6c+7lzrp9zbhBN1/ZLzrnpJEF2M0s3s8wdX9N0/8liuvg145z7BNhgZsObFx0HLKWL5/6MaXw6JAqSI/t64FAzCzf/vtnxfe/y17p4J5nbprZI5nZsTyVze9cWydo2tkU3aUfbIhnb3Lbo1PbZmm/a7bHM7BSa7svwA3c65672ONIXMrP7gElAPrAZ+F/gUeBBYACwDjjLObfVq4y7YmYTgdeARXx678sVNN3L1NWzHwj8g6brwwc86Jz7tZkNoemvwnnAAuAc51yjd0l3z8wmAT9xzp2aDNmbMz7S/DYA3Oucu9rMetH1r5mDgduBELAauJDma4cunBtaPiitB4Y456qal3X57zmANT3C5GyaZrpdAFxE0z07XfpaF+8kc9vUFt2lHdtTydbetUUyt41tkcztaFskc5vbFp3ZPvf4wlZERERERESSW08fiiwiIiIiIiJJToWtiIiIiIiIJDUVtiIiIiIiIpLUVNiKiIiIiIhIUlNhKyIiIiIiIklNha2IiIiIiIgkNRW2It2ImV1lZv/yOoeIiIg0Udss0jlU2IqIiIiIiEhSU2ErkqTM7GdmVmpm281suZl9FbgCONvMaszs/ebtss3sDjPb1Lz9b83M37zuAjN7w8z+ZmZVZvahmR3n5XmJiIgkK7XNIt4JeB1ARNrOzIYDlwJfcc5tNLNBgB/4HTDUOXdOq83vAsqAoUA68CSwAbi1ef0E4CEgHzgDeNjMBjvntnb8mYiIiHQPaptFvKUeW5HkFAdSgJFmFnTOrXXOrfrsRmZWBJwCXOacq3XOlQF/Aaa22qwMuM45F3XOPQAsB77a8acgIiLSrahtFvGQemxFkpBzbqWZXQZcBYwys2eBH+1i04FAENhkZjuW+Wj6q/AOpc451+r9OqBvu4cWERHpxtQ2i3hLPbYiSco5d69zbiJNDaQD/tD8b2sbgEYg3zmX0/zKcs6NarVNsbVqWYEBwMaOzC4iItIdqW0W8Y4KW5EkZGbDzexYM0sBGoB6IAFsBgaZmQ/AObcJeA74k5llmZnPzPYzs6Nb7a4Q+L6ZBc3sm8AI4OlOPSEREZEkp7ZZxFsqbEWSUwrwe2AL8AlNDeDPgTnN6yvM7L3mr88DQsBSoJKmySj6tNrXPKCkeV9XA2c65yo6+gRERES6GbXNIh6ynYfvi0hPYmYXABc1D5sSERERj6ltFtk76rEVERERERGRpKbCVkRERERERJKahiKLiIiIiIhIUlOPrYiIiIiIiCQ1FbYiIiIiIiKS1FTYioiIiIiISFJTYSsiIiIiIiJJTYWtiIiIiIiIJDUVtiIiIiIiIpLU/j9Fm8Y+VbA5fQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1152x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from nndl import plot\n",
    "\n",
    "paddle.seed(100)\n",
    "# 学习率大小\n",
    "lr = 0.005  \n",
    "# 批次大小\n",
    "batch_size = 64\n",
    "# 加载数据\n",
    "train_loader = io.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)\n",
    "dev_loader = io.DataLoader(dev_dataset, batch_size=batch_size)\n",
    "test_loader = io.DataLoader(test_dataset, batch_size=batch_size)\n",
    "# 定义网络，不使用残差结构的深层网络\n",
    "model = Model_ResNet18(in_channels=1, num_classes=10, use_residual=False)\n",
    "# 定义优化器\n",
    "optimizer = opt.SGD(learning_rate=lr, parameters=model.parameters())\n",
    "# 实例化RunnerV3\n",
    "runner = RunnerV3(model, optimizer, loss_fn, metric)\n",
    "# 启动训练\n",
    "log_steps = 15\n",
    "eval_steps = 15\n",
    "runner.train(train_loader, dev_loader, num_epochs=5, log_steps=log_steps, \n",
    "            eval_steps=eval_steps, save_path=\"best_model.pdparams\")\n",
    "# 可视化观察训练集与验证集的Loss变化情况\n",
    "plot_training_loss_acc(runner, 'cnn-loss2.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 5.4.2.2 模型评价\n",
    "\n",
    "使用测试数据对在训练过程中保存的最佳模型进行评价，观察模型在测试集上的准确率以及损失情况。代码实现如下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Test] accuracy/loss: 0.8100/0.5841\n"
     ]
    }
   ],
   "source": [
    "# 加载最优模型\n",
    "runner.load_model('best_model.pdparams')\n",
    "# 模型评价\n",
    "score, loss = runner.evaluate(test_loader)\n",
    "print(\"[Test] accuracy/loss: {:.4f}/{:.4f}\".format(score, loss))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，对比LeNet-5模型评价实验结果，网络层级加深后，训练效果不升反降。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.4.3 带残差连接的ResNet18\n",
    "\n",
    "#### 5.4.3.1 模型训练\n",
    "\n",
    "使用带残差连接的ResNet18重复上面的实验，代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Train] epoch: 0/5, step: 0/80, loss: 3.41889\n",
      "[Train] epoch: 0/5, step: 15/80, loss: 0.81710\n",
      "[Evaluate]  dev score: 0.53500, dev loss: 1.37067\n",
      "[Evaluate] best accuracy performence has been updated: 0.00000 --> 0.53500\n",
      "[Train] epoch: 1/5, step: 30/80, loss: 0.31470\n",
      "[Evaluate]  dev score: 0.81500, dev loss: 0.55274\n",
      "[Evaluate] best accuracy performence has been updated: 0.53500 --> 0.81500\n",
      "[Train] epoch: 2/5, step: 45/80, loss: 0.02694\n",
      "[Evaluate]  dev score: 0.87000, dev loss: 0.35305\n",
      "[Evaluate] best accuracy performence has been updated: 0.81500 --> 0.87000\n",
      "[Train] epoch: 3/5, step: 60/80, loss: 0.01007\n",
      "[Evaluate]  dev score: 0.86000, dev loss: 0.31423\n",
      "[Train] epoch: 4/5, step: 75/80, loss: 0.01729\n",
      "[Evaluate]  dev score: 0.87000, dev loss: 0.31510\n",
      "[Evaluate]  dev score: 0.86500, dev loss: 0.30383\n",
      "[Train] Training done!\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAF6CAYAAAAtT8R+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3Xl8nGW9///XZ/ZJ0iVt0zbd0z1hK9IWbJUdAUUQIvyKHhUPiKByUHn4BTnigkcRD6uI54jHczguB0SCLMoiyi5bw06blm50X9K9aWaf6/fHTEOabkk7mXuSvJ+Pxzw6c891z7yzNDOfua77c5tzDhEREREREZGeyud1ABEREREREZFDocJWREREREREejQVtiIiIiIiItKjqbAVERERERGRHk2FrYiIiIiIiPRoKmxFRERERESkR1NhKyIiIiIiIj2aClsRERERERHp0VTYioiIiIiISI+mwlZERERERER6tIDXAQ7FkCFD3Lhx47yOISIivcRrr7220TlX5XWOnkyvzSIiUkidfW3u0YXtuHHjaGxs9DqGiIj0Ema23OsMPZ1em0VEpJA6+9qspcgiIiIiIiLSoxWlsDWziJm9amZvmdk8M/vBXsZcZGbNZvZm/nJJMbKJiIiIiIhIz1aspcgJ4GTnXIuZBYEXzOwx59zLHcb9wTn3tSJlEhERERERkV6gKIWtc84BLfmbwfzFFeO5RUREREREpHcr2jG2ZuY3szeBDcCTzrlX9jKs3szeNrP7zWx0sbKJiIiIiIhIz1W0rsjOuQwwzcwGAn8ys8Odc++2G/IIcI9zLmFmXwb+Fzi54+OY2aXApQBjxowpQnIR6Uuy2SyrVq1i586dXkeRAgsGgwwdOpT+/ft7HUVEREQKrOin+3HObTWzp4EzgHfbbd/Ubth/AT/dx/53AXcBTJ8+XcuZRaSgNm7ciJkxZcoUfD41ju8tnHPEYjFWr14NoOJWRESklylWV+Sq/EwtZhYFTgMWdBhT3e7m2UBTMbKJiLS3detWhg0bpqK2lzEzysrKGDlyJBs2bPA6joiIiBRYsWZsq4H/NTM/uWL6Pufcn83seqDROfcw8C9mdjaQBjYDFxUpm4hIm0wmQzAY9DqGdJNoNEoqlfI6hoiIiBRYsboivw0cvZft3213/dvAt4uRR0Rkf8zM6wjSTfSzFRER6Z201k5ERERERER6tD5f2LpUhvgTi8ms3eF1FBGRkrVgwQLMjMbGxkN6nMcffxwzY+PGjQVKJiLiPecc6fXbyWxswaWzXscR2SfnHNlYksymFtJrtpLdmcC53tGPt+hdkUtNZuU2Np3xO/rf/DH6fXOW13FERA7KgZbYjh07lvfff/+gH3/SpEmsXbuWIUOGHPRjiIj0Rul124i/upTsphYAys89Bv/AMpKL1pNcsBZfWQgrC+X/DROsGYIF/LhMFnymQySkoLLxFNnNLWR3JnGxJNnWBG5nksiMGnz9o6QWrCX+8pLd9rFIkPJPHIWvf5Tsjjgum8XXP9rjfjf7fGEbGD+I4LThxBuaVNiKSI+1du3atusvvvgi9fX1vP7661RX5xrO+/3+ve6XTCYJhUIHfHy/38/w4cMLE1ZEpBdwiRSxfywivXwTVhYiMnsSFg7gqwgDYAEfFvKT3R4ju24bJNMABMflPiBMvL6cZNNqLBrGVx7CorniNzxzPGZGdnsM5xy+sjAW3PvfcOkbXDoDgAX8ZFsTpJY241rzRWtrkmxrkuiHJxIYWUlmw3Zif5//wc6hAL6yEC6R+/3zDx9A5NgJWFko9/u5NUZmcwtWnvu9Tc5fTXL+Ggj48Q8qxze4HP+gCoKThpV8odvnC1uASH0tO657msya7fhH6NyGItLztC86Bw0aBEBVVdUexejw4cO5/PLLWb16Nffffz9HHHEEzz77LDfddBO/+c1vWLJkCQMGDODkk0/mlltuYejQoUBuKXJtbS1z585l+vTpbbcbGhr41a9+xbPPPsuIESO4/vrr+cxnPtOl7C+88ALXXHMNjY2NlJWVcdZZZ3HrrbcyePBgAJYvX86VV17J888/T2trKyNHjuSKK67gyiuvBOD+++/nhz/8IYsWLSIcDjN16lR+9atfcfjhhx/091NEZF+cc7k3+MEArjVJ+OixhA4fiQV2Lz6DNVUEa6o+2C+dwcWSbUVqoHoAwAfFyeadZDZsJ3LsBADiry8nvaw5/2B+fGUhfAPKKDulDoDUqs2QzuxWGJu/zx9l2KO4TDY/q5rEFw3h6xch25ok0biMbGsS15ogG0tCMkNk1kRCU6pxsRSJucvA78utAigP46/qB6Hc75V/aH/KzjyybaVAx99Lf2U5/sryDzaMqNzt/uDUanyV5WQ2t5DdtJPUog2kgxsJTc69n4i/9j4ulsQ/qKKt6C2VD15U2ALR+jp2XPc0sT8toOKrM72OIyLSrW6++WauueYaXnnlFTKZ/KfAZtx2223U1NSwZs0avvGNb/C5z32OJ554Yr+PdfXVV3PjjTdyxx138J//+Z9cdNFFzJo1i3HjxnUqy8qVKzn99NM5//zz+eUvf8nGjRu57LLLmDNnDk8++SQAX/rSl/D7/Tz11FMMGDCAJUuWsGnTJgBWrFjBnDlzuPnmmzn77LOJxWK89tpr+5yhFhE5WC6bJbVgLckFayk/axoWClD2iaM6PYtlAT/WL9p2OzBqEIFRg/Y5Pnz4SAKjKnGtyVzhG0vuVrgm315JZv323fbxD+tP+cePAiDx1gpcJpub8d21FLo8jC964FU6cmhc1uHiqd1mVX39IwRGVOJSGXY++lbu5xr/4PRzoaNGE/nQOPDllrf7ykL4BpYRGFGJlYXwD+kHgK+ynH6f/TAE/Xv93fNFgviGDzjo7P4BZfgHlH3wtTiHiyU/uB1PkV6xidSi9W3bAmMGt33gkmnegfWL4IsU/9SJKmyBYG0VgdohxBrmq7AVkd1s/fpjpN5cV/TnDU4bzsDbzuyWx/7oRz/Ktddeu9u2q666qu16TU0Nt99+O7NmzWLTpk1tM6d7841vfIPzzjsPgB//+MfccccdPPvss50ubH/2s58xbNgw/uu//otAIPeSdPfdd3Pcccfx6quvMnPmTJYvX84Xv/hFjjoq92at/WOvXr2abDbLBRdc0Lbsuq6urlPPLSLSGc450qs2k5i7jOy2GP7qAbhkGgsFunVppn9Iv7ZiZm+iJ9fli95EfnYviYU++FAvtWIz2U07oF1foMCoSspOy61maX1yHhj5ojdX/PoHlbc9Z9vMtOwhs7UVtzOx21JgX/8o4cNG4pxjx/+9BKnMbvsEJw0jMKISAj58FRFsSL/cTHtZGIuG8FfmiklfJES/C/Zdj5jPIFS8Es7MsLJw2+3o7Em4WRNxrbkGVNnNLVg4V8Q659j5xDuQyhCqHUHkuAlFywkqbNtE6+vY8ePnyTTvxF9VfuAdRER6qJkz93zB/Nvf/saNN97IggUL2Lp1K9lsrqvn8uXL91vYTps2re16KBRiyJAhrF+/fp/jO5o3bx6zZs1qK2p35YtEIsybN4+ZM2fyzW9+k6997Ws89NBDnHjiiZx11lnMnj0bgBkzZnDCCScwZcoUTjvtNE488UTOO+88Ro4c2ekMIiL74lIZWv8+n8zarfj6R4meUkdg9KCSKPh8kSBEgjBo7+9bKz45LT9zmGwrvnZbMmqQbUngNmxvO/4yOGU40SH9cFnHjt+/iEWCH8z4RkMERg8iMLISl3Vkt8dyhVmwd5QTLpNtmxFPrdxEdmvsgw8NYkl8FRGix08BIPb3+WS3x9r2tXCAwNjcsdNmRvioMbljrNt9aGD5mXIza5vd7KnMLDf7Xx6GMe3eIzgoO7mWzKadbYV6MfWO38QCiNTXsuPfniP+0ALKLznG6zgiUiK6a9bUS+Xlu78JWrx4MWeddRaXXHIJP/jBDxg8eDBLlizhE5/4BMlkch+PktOx8ZSZtRXFhfLlL3+ZT3ziEzz++OM8/fTTnHbaaXzmM59pm+V96qmneOWVV/jb3/7Gvffey9VXX81DDz3EaaedVtAcItJ3uHQmt3Q4f2xr8NjxBKdWY76edQyr+fKzbWVhOh6gUXbqYW3XXSaLa02CL1+wZ7OEpo5o66qb3bKT9OotWCSYK2xjSXb+6bXc2IC/7XjOUN1IgmMH45Lp3PhdS6CjYSzgzfdu19fmEqm22ejke+vIrNu2+4xrNERF/fTc/fNWk1m7LTe72qEwBYjMmgg+y90XDe3xtYWPGFW8L7CEmM8IjKjMzUx7QIVtXvCo4fjHVxJraFJhKyJ9yiuvvEIqleK2225rmzn9xz/+UZTnPuyww2hoaCCdTrc996uvvko8Ht+t+dOoUaO45JJLuOSSS7j77rv553/+Z+68807C4TBmxnHHHcdxxx3Hd77zHU488UTuvvtuFbYi0mUunSE5bzXJeasp/+Q0fP2ibbN0vZn5fVi/yAe3A34iM2r2GLfrfKcW8hM9fsoHDY7yS6HJ35/Z1krsmQW7P0c4QOQjkwmOGUxmW4zU4vXtToX0QfFovs7NhrfNRu9Mts2qBqcMx8xIvLuK1OL1+YI2NxuN30e/z83CzMg07yC9fhu+sjC+yvLccaz9P/j6o8dPxYK+fc5GB6oHdiqjFJcK2zwzI1pfS8ttL5PdGsM3MHrgnUREeoHJkyeTzWa59dZb+fSnP83rr7/ODTfcUJTnvvLKK/nFL37BJZdcwre+9S02btzI5ZdfzqmnnsqMGTMAuOyyy/jUpz7FpEmTiMViPPjgg0yYMIFwOMwzzzzDiy++yKmnnsrw4cNZsGAB8+fPV1ErIl3inCO9tDnX8XVngsDYwVACy41Lza4l2BYMEJwwdJ/j/JUVlH/qQ22zobuK312nQspubyX5zsrdjv8FKDvjCALVA0mv2UJy/pq2GV8wsq0JIseMw8JBEm+vJPH6+3vsH6wZAuEgtus41mED8EVD7R4nJzp70n6/zvZjpedQYdtOtL6Oln9/kfgj71H2uaO8jiMiUhQzZszglltu4eabb+a73/0uxx57LLfeeiuf/OQnu/25R40axRNPPMHVV1/NMcccs9vpfnbJZDJcccUVrFq1irKyMmbNmsUjjzwCQGVlJc899xy33347W7dupbq6mosvvpirr76627OLSO/gso7Wx94ms2E7vsEVRI+fQuAQuspK7hy+/spyqNz78b/B0YMJfP4jH3QOjuU7Bw/MHZfpUhmyOxO45h1tnYMtHCB72Ej84SD+qn6Ejhy9e8fnsnBbU6XQ1BGEpo4ozhcrJcN2LSnoiaZPn+4aGxsL9ngum2X92NsIHlPN4AcvLNjjikjP0dTURG1trdcxpBvt72dsZq8556YXOVKvUujXZpHuko0l2059k3hzOVYRIThhaEk0hpIPuEyub4PO0dt3dfa1Wb8h7ZjPR+S8WuJPLCHbkvA6joiIiIgUmEukic9dSst9r5Jevw2A8LSxhCYOU1FbgszvU1ErnaLfkg6i9bUQTxN/dJHXUURERESkQFzWkWxaQ0vDXJLvriY4fii+dg2TRKRn0zG2HYRmj8E3tJx4QxNlFxx+4B1EREREpKQ552h99C0yzTvwDx9AZMZ4/EMqvI4lIgWkGdsOzO8jcu5U4n95DxdLeR1HRETkkJnZGWa20MwWm9k1e7l/jJk9bWZvmNnbZvbx/PZxZhYzszfzl/8sfnqRg5fZ1opzDjMjOKWa6Ml1lJ1xhIpakV5Ihe1eRM+rxe1MEf/rEq+jiIiIHBIz8wN3AmcCdcCFZlbXYdh3gPucc0cDc4BftLtviXNuWv5yWVFCixyibCxJ7B+L2Pmn10gvbQYgNGkYwbGDdRytSC+lwnYvwifVYJURYg3zvY4iIiJyqGYCi51zS51zSeBe4JwOYxzQP399ALCmiPlECsalMyTeXknL/Y2kFq0nVDeSwKhKr2OJSBHoGNu9sKCf6NlTiD24AJdMYyF9m0REpMcaCaxsd3sVcGyHMd8H/mpmVwDlwKnt7qsxszeA7cB3nHPPd2NWkUPS+rd5ZNZuIzBmMOHpNfgHRL2OJCJFohnbfYjU1+G2JUg8tczrKCIiIt3tQuBu59wo4OPAb83MB6wFxuSXKH8T+D8z699xZzO71Mwazayxubm5qMFF0hu249IZAMJHjqbsjCMoO6VORa1IH6PCdh8ip43HKkLEGpq8jiIiInIoVgOj290eld/W3sXAfQDOuZeACDDEOZdwzm3Kb38NWAJM7vgEzrm7nHPTnXPTq6qquuFLENlTdkec1qebaP3LWySb1gIQGFFJoHqgx8lExAsqbPfBIkEiZ00m/uCCtk8BRURk/y666CJOPfXUAw+UYpoLTDKzGjMLkWsO9XCHMSuAUwDMrJZcYdtsZlX55lOY2XhgErC0aMlF9sIl08TnLqPlgUbSKzcTmjaG0NRqr2OJiMdU2O5HtL6W7MZWks+v8DqKiMgBXXTRRZhZ7rQWwSBDhgzhIx/5CD/96U/ZuXOn1/HEI865NPA14AmgiVz343lmdr2ZnZ0fdhXwJTN7C7gHuMg554DjgbfN7E3gfuAy59zm4n8VIh+IPf8eyXdXERxfRUX9dCJHj8WCfq9jiYjH1BVpP8JnTsKiAWIN8wmfVON1HBGRA/roRz/KfffdRzabZdOmTbzwwgvccMMN/PrXv+a5555j2LBhXkcUDzjnHgUe7bDtu+2uzwdm72W/BqCh2wOKHEB69RZ8lWX4ysKEPzSW8FGj8Q/p53UsESkhmrHdD195iPAZE4k90ITLZr2OIyJyQKFQiOHDhzNixAiOOOIILr/8cl566SWam5u55pprdht7xx13MHXqVCKRCJMmTeJHP/oR6XQagH/9139lypQpezz+5Zdfzkc+8pFO53HOcdNNNzF+/HhCoRATJkzgtttu223MQw89xNFHH01ZWRkDBw5k5syZvPHGGwCkUim++c1vMmrUKMLhMNXV1cyZM6er3xYR6aEyW3ay86/v0vrXd0nOy52Fyl9ZrqJWRPagwvYAovV1ZNe2kHx5lddRREQOysiRI/nsZz/LAw88QDb/Id33v/99brrpJm644Qaampq4/fbb+eUvf8kPfvADAL7whS/w3nvv8corr7Q9TiKR4A9/+AOf//znO/3cv/jFL7juuuu45pprmDdvHt/61re45ppr+PWvfw3AunXrOP/887nwwguZN28eL730El//+tcJBHILiu644w7uu+8+fve737Fo0SIefvhhjjvuuEJ9a0SkRGVjSWIvLmLnQ6+Tad5BeOZ4wh8a63UsESlhWop8AJGzJkPQR7yhifCsMV7HEREP7Hzs7T22BccNIVQ7ApfO0PrkvD3vnziM0KRhZOMpYk/v2V09NKWa4Pgqsi0JYs8v3OP+8jOPLEz4vMMOO4zt27ezceNGKioq+OlPf8oDDzzAGWecAUBNTQ3/9m//xr/8y7/wwx/+kMmTJ3Psscfym9/8hmOPzZ3y9JFHHiEWi3HBBRd0+nl/8pOfcMUVV3DppZcCMGnSJBYuXMiPfvQjLr74YtauXUsqleKCCy5g3LhxANTW1rbtv3z5ciZPnswJJ5yAmTFmzBhmzJhRoO+KiJSqxOvLSS1aT2jqCELTxuCLBL2OJCIlTjO2B+AbECF82gRiDfPJ9dEQEel5dv39MjPmzZtHLBajvr6eioqKtsuXv/xltm3bxq7zkH7hC1/gD3/4A6lUCoDf/OY3nH322Qwc2LlTaWzfvp1Vq1Zx/PHH77b9hBNO4P3336e1tZUjjzyS008/ncMPP5xzzz2X22+/nZUrV7aN/eIXv8g777zDxIkTueyyy2hoaCCZTBbiWyIiJcQ5R2ppM5nNLQCEjx5D+bkfInLcBBW1ItIpmrHthGh9LVsfXUTq9bWEjhnhdRwRKbL9zZ5awL/f+32R4P7vrwgXfHZ2b+bNm8eAAQMYPHgwS5fmztbyxz/+kcmT9zglKYMGDQJgzpw5fP3rX+cvf/kLs2fP5vHHH+fBBx8saC6/389jjz3G3Llz+dvf/kZDQwPXXHMNf/zjHznrrLOYNm0ay5Yt48knn+Tpp5/myiuv5LrrruPll1+mf//+Bc0iIt5Ib9hO4tWlZJp3EJwynOisSfjKwl7HEpEeRjO2nRA5Zyr4jVjDfK+jiIh02erVq/n973/Peeedh8/n47DDDiMSibB06VImTpy4x8Xvz502o7Kykk9+8pP89re/5Z577mHQoEGcfvrpnX7e/v37M2rUKJ577rndtj/77LPU1NRQVlYG5GaRZ86cybXXXstzzz3HCSecwP/8z/+0ja+oqODcc8/lZz/7GY2NjTQ1NfHss88W4DsjIl7KtsRpfXYBrX95i2xLnMjsSUSOm+h1LBHpoTRj2wn+wWWETxxHvKGJ/j86BTPzOpKIyF4lk0nWrVu3x+l+hg4dyg033ADkCsVrr72Wa6+9FjPj1FNPJZ1O88477/DGG29w4403tj3e5z//ec4//3yampr47Gc/21b0dta3v/1trrrqKiZNmsSJJ57IU089xX/8x39w5513AvDiiy/y97//nY997GNUV1ezaNEi3n77bS6++GIA/v3f/50RI0Ywbdo0ysrKuOeee/D7/XudaRaRniW1aD3p5ZsIHTWa8BGjdS5aETkkKmw7KVJfx7av/IX0vA0ED9d5IEWkND3//PNUV1fj9/sZMGAAtbW1fO1rX+OrX/0q5eXlbeOuu+46qqur+fnPf85VV11FNBpl8uTJXHTRRbs93plnnsmAAQNoamrinnvu6XKeyy+/nJ07d/LjH/+Yr3zlK4wePZqf/OQnbYXrgAEDeOmll7jzzjvZsmULw4cP57Of/SzXXXcdkJv1veWWW1i0aBHZbJba2loaGhr2eioiESltLutILVqHrzxMYNQgQoePIjhpOL4KLTsWkUNnPbkh0vTp011jY2NRniuzbgfrRtxMv++dSP/vnViU5xSR4mtqatqtK6/0Pvv7GZvZa8656UWO1KsU87VZeo706i3EX11KdmsrwQlDiR6vD6dEpHM6+9qsY2w7yT+8H6HZY3ScrYiIiEgnZba20vrku7T+9V1cJkv0pFoiH9WhBCJSeEUpbM0sYmavmtlbZjbPzH6wlzFhM/uDmS02s1fMbFwxsnVFtL6W9DsbSC/a5HUUERERkZKXad5BesN2wjNqqDj3GILjhqhXiYh0i2LN2CaAk51zRwHTgDPM7LgOYy4GtjjnJgK3AjdSYiLn5ZauadZWREREZE8ukyXxziqSC9cCEJw4lIr6GYQPH4X5tVBQRLpPUf7CuJyW/M1g/tLx4N5zgP/NX78fOMVK7CO9wJiBBGeMINbQ5HUUERERkZLhnCO1rJmWB14j0biMzPrtQO50Xr5I0ON0ItIXFO2jMzPzm9mbwAbgSefcKx2GjARWAjjn0sA2YHCx8nVWtL6OVOMa0su3eh1FRERExHOZzS20Pvo2sWcWYEEfZR87XM2hRKToilbYOucyzrlpwChgppkdfjCPY2aXmlmjmTU2NzcXNmQnROtzy5HjD2jWVqS36snd4mX/9LMVKTyXTJPdESMyexLlZ3+IwMhKryOJSB9U9IMdnHNbgaeBMzrctRoYDWBmAWAAsEeXJufcXc656c656VVVVd0ddw+BiYMJHDlMx9mK9FJ+v59UKuV1DOkmsViMYFDLIkUOhUulib/2PvG5ywAIDB9IxadnEJo8HPOV1FFkItKHFKsrcpWZDcxfjwKnAQs6DHsY+EL++qeBp1yJfrQera8l+eJKMmt3eB1FRAps4MCBrF+/nmw263UUKSDnHK2traxevZqhQ4d6HUekR3JZR3LhWloaGkm+vRIXT7atgrCA3+N0ItLXBYr0PNXA/5qZn1wxfZ9z7s9mdj3Q6Jx7GPg18FszWwxsBuYUKVuXRevr2PG9Z4j9qYmKr8z0Oo6IFNCQIUNYtWoVCxcu9DqKFFgwGGTYsGH079/f6ygiPU5mYwuxFxaS3dKKf2h/wqfUEajS/yURKR1FKWydc28DR+9l+3fbXY8D5xcjz6EK1FURmDKYeIMKW5HexufzMWbMGK9jiIiUBOccZoaF/JB1RE+cSkDnohWREqQTih0EMyNSX0fi2ffJbNzpdRwRERGRgsrGU8ReXkzsqVyzTF//KOXnHkOwpkpFrYiUJBW2BylaXwsZR/whLVcUERGR3sFlsiTeXUVLw1xSC9ZiZSFcNn8crQpaESlhxTrGttcJHl2Nf9xAYg3zKb/4Q17HERERETkkmc0ttD7VhNsRJzCykvDMGvwDy72OJSLSKSpsD5KZEa2vpeVnr5DdFsc3IOJ1JBEREZEuc+kMFvDjK4/gKw8T/vBEnYtWRHocLUU+BJH6Okhlif/5Pa+jiIiIiHRJtiVB7LmF7PzLW7isw8IBys88UkWtiPRIKmwPQejYkfhG9CPWMN/rKCIiIiKd4lIZ4q+/T8sDjaTeb84VsvnjaEVEeiotRT4E5vMRPa+W1l+/TnZnEl95yOtIIiIiIvuUXr2F2PMLcbEUgZoqIseMw9dPh1OJSM+nGdtDFK2vxcXSJB5b5HUUERERkf3yDSondNgoyj5xFGUnTlVRKyK9hgrbQxT66Fh8VWXEGpq8jiIiIiKyX75oiPARowgM7e91FBGRglJhe4jM7yPyqanE//weLp7yOo6IiIjIXiUXrSe5aL3XMUREuoUK2wKI1tfhWpLEn1zqdRQRERGRPTjnSLy5gtSyZq+jiIh0CxW2BRA+aRw2MEJc3ZFFRESkBGWad+Ba4gTHV3kdRUSkW6iwLQALBYicPYXYwwtxqYzXcURERER2k17aDH4fwTGDvY4iItItVNgWSLS+FrclTuLpZV5HEREREWnjsi53vtpRlVhIZ3oUkd5JhW2BRD42AasIqTuyiIiIlBQXS2JlYYLjh3odRUSk26iwLRCLBIl8YhLxBxfgMlmv44iIiIgA4CsPU3H20QTGahmyiPReKmwLKFpfR3bDTpIvrPA6ioiISBszO8PMFprZYjO7Zi/3jzGzp83sDTN728w+3u6+b+f3W2hmpxc3uRwql8niUmkAzMzjNCIi3UeFbQGFz5wIkQAxdUc4JvvxAAAgAElEQVQWEZESYWZ+4E7gTKAOuNDM6joM+w5wn3PuaGAO8Iv8vnX524cBZwC/yD+e9BDpVZvZcc8rZDa3eB1FRKRbqbAtIF9FmMgZE4k90ITLajmyiIiUhJnAYufcUudcErgXOKfDGAf0z18fAKzJXz8HuNc5l3DOLQMW5x9PeojU0mYs6Mc3sNzrKCIi3UqFbYFF62vJrt5B6tXVXkcREREBGAmsbHd7VX5be98H/snMVgGPAld0YV8pUS6ZJr1yM4FxQzCfliGLSO+mwrbAImdNhqBP3ZFFRKQnuRC42zk3Cvg48Fsz6/R7BDO71Mwazayxubm520JK16RWbIJMluD4Kq+jiIh0OxW2BeYbGCV8ynhiDfNxznkdR0REZDUwut3tUflt7V0M3AfgnHsJiABDOrkvzrm7nHPTnXPTq6pURJWK9NJmrDyMf2j/Aw8WEenhVNh2g2h9LZllW0m9uc7rKCIiInOBSWZWY2Yhcs2gHu4wZgVwCoCZ1ZIrbJvz4+aYWdjMaoBJwKtFSy6HJDyjhujsSeqGLCJ9ggrbbhA5Zyr4jLi6I4uIiMecc2nga8ATQBO57sfzzOx6Mzs7P+wq4Etm9hZwD3CRy5lHbiZ3PvA48FXnXKb4X4UcDH9lOYGRlV7HEBEpioDXAXojf1U5oRPGEmtoov+/neJ1HBER6eOcc4+SawrVftt3212fD8zex74/An7UrQGl4BJvrsA/fACB4QO8jiIiUhSase0m0fo60gs2kpq/wesoIiIi0odkWxIk3lhOZt02r6OIiBSNCttuEj13KoC6I4uIiEhRpZblOlOrG7KI9CUqbLuJf0R/QrNG6zhbERERKarU0g34hvTD1z/qdRQRkaJRYduNIvW1pN5aT3rJZq+jiIiISB+Q2dZKdvNOzdaKSJ+jwrYbRc+rBSCmWVsREREpAteSwMrDBGuGeB1FRKSoVNh2o8C4SoLHVOs4WxERESmKwMhKKs6fga8s7HUUEZGiUmHbzaL1daReXU16pToTioiISPdxqTQu6zAzr6OIiBSdCttuFqnPLUeOP6BZWxEREek+iTdW0HL/XFwm63UUEZGiU2HbzYKThxA4fKiOsxUREZFu45wjtawZ/6ByzK+3dyLS9xTlL5+ZjTazp81svpnNM7Mr9zLmRDPbZmZv5i/fLUa2YojW15J8YQWZdTu8jiIiIiK9UGb9NlxrUt2QRaTPKtZHemngKudcHXAc8FUzq9vLuOedc9Pyl+uLlK3bRevrwEH8wQVeRxEREZFeKLWkGQI+AqMHex1FRMQTRSlsnXNrnXOv56/vAJqAkcV47lIQOHwo/kmD1B1ZRERECs5lsqSXbyQwZjAW9HsdR0TEE0U/CMPMxgFHA6/s5e4Pm9lbZvaYmR1W1GDdyMyI1teReHoZmU2tXscRERGR3sSM6IlTCR8+yuskIiKeKWpha2YVQAPwdefc9g53vw6Mdc4dBdwBPLiPx7jUzBrNrLG5ubl7AxdQtL4WMo74wwu9jiIiIiK9iPmMwIhK/IMrvI4iIuKZohW2ZhYkV9T+3jn3QMf7nXPbnXMt+euPAkEzG7KXcXc556Y756ZXVfWcBgnBY0bgHztA3ZFFRESkYFw6Q3zuMrLbY15HERHxVLG6Ihvwa6DJOXfLPsYMz4/DzGbms20qRr5iMDMi59WSeHIp2e1xr+OIiIhIL5BesZnku6vI7kx4HUVExFPFmrGdDXwOOLnd6Xw+bmaXmdll+TGfBt41s7eAnwFznHOuSPmKIlpfB8kM8T+/53UUERER6QVSy5qxshD+YQO8jiIi4qlAMZ7EOfcCYAcY83Pg58XI45XQh0fhq64g1tBE2WeO9DqOiIiI9GAukSK9ajOh2hGYb79vs0REer2id0Xuy8znI3puLYnHFpHdmfQ6joiIiPRgqeWbIOsIju85PUdERLqLCtsii9bX4mJpEo8v9jqKiIiI9GAukcI3qByfuiGLiKiwLbbQ8WPxDY6qO7KIiIgckvARoyk/+2jyvTdFRPo0FbZFZgE/kU9NJf7n93CJtNdxREREpAdyydx7CBW1IiI5Kmw9EK2vw+1IEn9yiddRREREpAdq/eu7tD6zwOsYIiIlQ4WtB8Kn1GADwsQbmryOIiIiIj1MdkeMTPMO/IPLvY4iIlIyVNh6wEIBIp+cQuyhBbhUxus4IiIi0oOkljYDEKxRN2QRkV1U2HokWl+L2xIn8cz7XkcRERGRHsI5R2ppM/5h/fFVRLyOIyJSMlTYeiRy+kSsPKjuyCIiItJp2S2tZLe2arZWRKQDFbYesWiQyMcnEf/TAlwm63UcERER6QF8/SJEPjqZQM0Qr6OIiJQUFbYeitTXkd2wk+Q/VngdRURERHoAC/oJTRyGLxLyOoqISElRYeuhyMcnQdhPTN2RRURE5AAym1pIvLuq7Ry2IiLyARW2HvL1CxM5fSLxB5pwWS1HFhERkX1LLlpH4vXlXscQESlJKmw9Fq2vJbNqO6nGNV5HERERkRLlso70so0ERlVioYDXcURESo4KW49FPjkFAj4tRxYREZF9yqzdiounCI4f6nUUEZGSpMLWY77KKOFTaog1zMc553UcERERKUGppc0Q9BMYNcjrKCIiJUmFbQmI1teRWbKF9NvrvY4iIiIiJcil0gTHDcECeusmIrI3+utYAiKfmgo+I9Yw3+soIiIiUoLKTq4jMnuS1zFEREqWCtsS4K8qJ3T8WB1nKyIi3cLMzjCzhWa22Myu2cv9t5rZm/nLe2a2td19mXb3PVzc5ALgUhkAzMzjJCIipUuFbYmI1teSnt9MakGz11FERKQXMTM/cCdwJlAHXGhmde3HOOe+4Zyb5pybBtwBPNDu7tiu+5xzZxctuADgkml23PsKyfk6e4KIyP6osC0R0XNrAYhr1lZERAprJrDYObfUOZcE7gXO2c/4C4F7ipJMDii1YhOkM/gGV3gdRUSkpKmwLRH+kf0JfXiUjrMVEZFCGwmsbHd7VX7bHsxsLFADPNVuc8TMGs3sZTP7VPfFlL1JL23GKsL4h/bzOoqISElTYVtCIvV1pN5YR3rpZq+jiIhI3zQHuN85l2m3baxzbjrwGeA2M5vQcSczuzRf/DY2N+uQmkLJxpOk12whWFOl42tFRA5AhW0JiZ6XW44ce0DLkUVEpGBWA6Pb3R6V37Y3c+iwDNk5tzr/71LgGeDojjs55+5yzk13zk2vqqoqRGYB0u9vBAfB8UO9jiIiUvJU2JaQQE0lwQ9VqzuyiIgU0lxgkpnVmFmIXPG6R3djM5sKVAIvtdtWaWbh/PUhwGxAx8wUSWD0YCKzJuKrLPM6iohIyVNhW2Ki9bWkXl5FZtU2r6OIiEgv4JxLA18DngCagPucc/PM7Hoza9/leA5wr3POtdtWCzSa2VvA08BPnHMqbIvEVx4mNKVay5BFRDoh4HUA2V3kvFq2/+tTxP60gIorjvU6joiI9ALOuUeBRzts+26H29/fy34vAkd0azjZq9TyjZDOEhiv42tFRDpDM7YlJji1ikBdlboji4iI9GGJN1eQmL9GRa2ISCepsC1B0fpaks+vILOhxesoIiIiUmSZra1kN+8kOF6NuEREOkuFbQmK1tdB1hF/cIHXUURERKTIUstyp0wK1qiwFRHpLBW2JShw5DD8EyrVHVlERKSPcc6RXrIBf/UAfGUhr+OIiPQYKmxLkJkRra8j8dQysltiXscRERGRInHxFPh9OnetiEgXqbAtUdH6WkhniT280OsoIiIiUiS+aIjyT32I4MRhXkcREelRVNiWqOCMkfhH9yeu7sgiIiJ9gnMOl85gZphP3ZBFRLpChW2JMjMi59US/+sSsjsSXscRERGRbpZZv40d975Cunm711FERHqcohS2ZjbazJ42s/lmNs/MrtzLGDOzn5nZYjN728w+VIxspSxaXweJDPG/vOd1FBEREelmqSXN4Bz+ynKvo4iI9DjFmrFNA1c55+qA44CvmlldhzFnApPyl0uB/yhStpIVmjUa37BydUcWERHp5VwmS3r5RgJjBmMBv9dxRER6nKIUts65tc651/PXdwBNwMgOw84BfuNyXgYGmll1MfKVKvP7iJ5bS+LRRWRbk17HERERkW6SXrMVl0irG7KIyEEq+jG2ZjYOOBp4pcNdI4GV7W6vYs/it8+J1NfiWlMknljidRQRERHpJqmlG7BwgMCIgV5HERHpkYpa2JpZBdAAfN05d1CdEczsUjNrNLPG5ubmwgYsQeETxmGDosTUHVlERKTXCh82isiHJ2J+9fUUETkYRfvraWZBckXt751zD+xlyGpgdLvbo/LbduOcu8s5N905N72qqqp7wpYQC/qJnjOF+CPv4RJpr+OIiIhIN/APqSBY0/vf14iIdJdidUU24NdAk3Puln0Mexj4fL478nHANufc2mLkK3XR+jrc9gSJvy/1OoqIiIgUWLJpDen1OsWPiMihKNaM7Wzgc8DJZvZm/vJxM7vMzC7Lj3kUWAosBn4FfKVI2Upe+NTxWP+wuiOLiIj0Mi6RIv7qUtLLN3odRUSkRwsU40mccy8AdoAxDvhqMfL0NBYOEDlrMvGHFuB+eZZOAyAiItJLpJZvgqwjOF7LkEVEDoU6FPQQ0fpasptiJJ5d7nUUERERKZDU0g34+kXwDa7wOoqISI+mwraHCJ8xESsLEld3ZBERkV4h25oks3YbgQlDybUjERGRg6XCtofwlYUInzmR2J8W4LJZr+OIiIjIIcpua8UiQXVDFhEpABW2PUi0vo7suhaSL670OoqIiIgcokD1QCr+v2PxDyzzOoqISI+nwrYHiXxiEoT86o4sIiLSw7l0Fucc5tMSZBGRQlBh24P4+keIfGwC8QeayDWRFhERkZ4oOW8VLffPxaUyXkcREekVVNj2MJH6WjIrtpFqXON1FBERKRLL+ZKZPWVmb+e3HW9mF3idTbrOOUdqaTO+8jAW1Cn8REQKQYVtDxM9ewoEfMTUHVlEpC+5HrgYuAsYk9+2Crjas0Ry0LJbWslubdW5a0VECkiFbQ/jG1RG+KRxxBq0HFlEpA+5CDjLOXcvsOuP/zJgvGeJ5KCllm4Ag8C4IV5HERHpNVTY9kDR+joyizeTfme911FERKQ4/EBL/vquwrai3TbpIZxzpJY1ExhRiS8S8jqOiEivocK2B4p8aioY6o4sItJ3PAbcYmZhyB1zC/wQeMTTVHJQIsdOIHTkaK9jiIj0Kp0ubM3sJDOryV+vNrP/NbP/MbPh3RdP9sY/rILQR8fqOFsRkb7jG0A1sA0YQG6mdiw6xrbHMTOCYwYTGD7A6ygiIr1KV2ZsfwHs6kl/MxAEsuQaWUiRRetrSc9rJrVwo9dRRESkG+VnZ4cA55NrHHUcMME5d65zboen4aRLXNaReHMF2R1xr6OIiPQ6XSlsRzrnVphZADgduBS4HJjVLclkv6Ln1QIQ16ytiEiv5nKdAt8Bss65Dc65uc65dV7nkq7LrN1K4o3lZDbr0GgRkULrSmG73cyGAScA851zu/4qBwsfSw7EP2oAwWNH6jhbEZG+4Q1gstch5NCklm6AoJ/AyEFeRxER6XUCXRh7BzAXCAFfz2+bDSwodCjpnGh9Hdv/35Okl20hUFPpdRwREek+zwCPm9ndwEo+6IyMc+6/PcokXeDSWVLLNxEcOwQLqHeniEihdfovq3PuRuBUYHb+PHoAq4FLuiOYHFi0PrccOfaAZm1FRHq52eTOW3sC8E/A5/KXf/IylHReetVmSGUIjq/yOoqISK/UlRlbnHPv7bpuZieRO97n2YKnkk4JjB9EcNpw4g3z6XeVDnUWEemtnHMneZ1BDk22JY5VhPFXD/Q6iohIr9SV0/08a2az89evBu4F/s/Mru2ucHJgkfpaki+tIrN6u9dRRESkG5lZpZl93sy+nf+308egmNkZZrbQzBab2TV7uf9WM3szf3nPzLa2u+8LZrYof/lCob6eviZ8+Cgq6mdgPvM6iohIr9SVgzwOB17OX/8ScBK5Uw5cVuhQ0nnR+joAYn/ScmQRkd7KzD4MLCH3mnsk8GVgSX77gfb1A3cCZwJ1wIVmVtd+jHPuG865ac65aeR6ajyQ33cQ8D3gWGAm8L2uFNSS49JZABW1IiLdqCuFrQ9wZjYBMOfcfOfcSkAvcB4K1lYRqB2i7sgiIr3bbcBXnHOznHMXOudmkzvl3s86se9MYLFzbqlzLkluxdU5+xl/IXBP/vrpwJPOuc3OuS3Ak8AZB/1V9FGtT82n9Wm9TouIdKeuFLYvAD8HbgL+BJAvcjd2Qy7pgmh9HcnnlpNp3ul1FBER6R6Tgfs6bLsfmNiJfUeS66S8y6r8tj2Y2VigBniqq/vK3mXjSTJrtuDrF/E6iohIr9aVwvYiYCvwNvD9/LapwO2FjSRdFamvhawj/pDOvCQi0kstAuZ02HY+ueXJhTQHuN85l+nKTmZ2qZk1mlljc3NzgSP1bOllG8FBcPxQr6OIiPRqne6K7JzbBFzbYdtfCp5Iuix41HD84yuJNTRRfskxXscREZHC+zrwZzP7F2A5MA6YBJzViX1XA6Pb3R6V37Y3c4Cvdtj3xA77PtNxJ+fcXcBdANOnT3cd7+/LUsua8Q0swz+o3OsoIiK9Wle6IgfN7AdmttTM4vl/f2Bmoe4MKAdmZkTra0n8fSnZrTGv44iISIE5514EJpA7JOg1cg2eJua3H8hcYJKZ1eRfs+cAD3ccZGZTyfXNeKnd5ieAj+U7MlcCH8tvk07ItsTJrN+uc9eKiBRBV5Yi/xQ4lVxHxqPy/54M3NgNuaSLovV1kMoSf+S9Aw8WEZEexcxGAjjnfuec+6lz7nfkGjqOONC+zrk08DVyBWkTcJ9zbp6ZXW9mZ7cbOge41znn2u27GfghueJ4LnB9fpt0goUDRGZNJDhBy5BFRLpbp5cikzuW56j8kmSAhWb2OvAW8I2CJ5MuCc4YgX9Uf2IN8yn73FFexxERkcJ6EPhnYEu7baOA/yJ3Kp79cs49CjzaYdt3O9z+/j72/W/gv7sWVwAsGCA0pdrrGCIifUJXZmz3dfI1nZStBJjPR+S8WuJPLCHbkvA6joiIFNZk59w77Tfkb0/1KI8cQHZ7jGTTGlwy7XUUEZE+oSuF7R+BR8zsdDOrNbMzyH2C/MfuiSZdFa2vhXia+KOLvI4iIiKF1Wxmu53aJ3970z7Gi8eSSzYQf2UJLp31OoqISJ/QlcL2/wF/A+7kg8YVTwPf6oZcchBCs8fgG1pOvEEngRcR6WX+G2gws7PMrM7MPgk0kFuKLCXGOUd6yQb8wwfiK1OPTRGRYtjvMbZmdnKHTc/kLwbsai7xET44kbt4yPw+Ip+aSuz3b+NiKSwa9DqSiIgUxk+AFHATuVP3rCBX1N7qZSjZu+zGFrI74kSOHH3gwSIiUhAHah71631s31XU7ipwxxcskRySaH0trXe9RvyvS4ieo0OvRER6iROA+51z/25m1eTOSHA4MBRY52ky2UNqWTP4jODYwV5HERHpM/Zb2DrnaooVRAojfFINVhkh1jBfha2ISO/xC+D0/PWb8/+mgLuAs/e6h3gmuzNBYNQgLKyVUyIixdKV0/1ID2BBP9GzpxB7cAEumcZC+hGLiPQCI51zK8wsAJwBjAGSwBpvY8nelJ1Ui8uqaZSISDF1pXnUQTOz/zazDWb27j7uP9HMtpnZm/nLd/c2TjonUl+H25Yg8dQyr6OIiEhhbDezYeSWJM9zzrXkt2tKsMS4TK6gNV9R3mKJiEhesf7q3k3uE+b9ed45Ny1/ub4ImXqtyGnjsYoQMXVHFhHpLe4A5gK/J3d2AoDZwALPEskeXCZLyx9fJfHuKq+jiIj0OUUpbJ1zzwGbi/FcAhYJEjlrMvEHF+DSGa/jiIjIIXLO3QicCsx2zt2b37wauMS7VNJRes0WXCyFb0CZ11FERPqcUlon82Eze8vMHjOzw7wO09NF62vJbmwl+fwKr6OIiEgBOOfec84t6XD7HS8zye5SS5uxcIDAiIFeRxER6XNKpbB9HRjrnDuK3HKrB/c10MwuNbNGM2tsbm4uWsCeJnzmJCwaINYw3+soIiIivZ5LZUiv2ERg7BDMXypvr0RE+o6S+MvrnNu+qxGGc+5RIGhmQ/Yx9i7n3HTn3PSqqqqi5uxJfOUhwmdMJPZAkzozioiIdLP0yk2QzhIcr/cmIiJeKInC1syGm5nlr88kl2uTt6l6vmh9Hdm1LSRfVhMLERGR7uQf2p/w9HH4hw3wOoqISJ9UlJOcmtk9wInAEDNbBXyP/CkKnHP/CXwauNzM0kAMmOOcc8XI1ptFzpoMQR/xhibCs8Z4HUdERKTX8lVECB8x2usYIiJ9VlEKW+fchQe4/+fAz4uRpS/xDYgQPm0CsYb59L/pY+QnxUVERKSA0mu24FIZAmMG67VWRMQjJbEUWbpPtL6WzPJtpF5f63UUERGRXinx9koSje97HUNEpE9TYdvLRc6ZCn5Td2QREZFukG1NkFm7jcD4Ks3Wioh4SIVtL+cfXEb4xHHEG5rQYcsiIiKFlVq2EUDdkEVEPKbCtg+I1NeRfm8T6XkbvI4iIiLSq6SWNuMbVI5/QJnXUURE+jQVtn1A9NypYBBraPI6ioiISK/hUmlcIkVw/FCvo4iI9HkqbPsA//B+hGaP0XG2IiIiBWTBABX10wnVjfA6iohIn6fCto+I1teSfmcD6UWbOr1Pev02Yi8t1rG5IiIiHTjncNksZob59XZKRMRr+kvcR0TOqwXo0qxtZt02UgvWEn9Rxa2IiEh72S07abn3FdLrtnodRUREUGHbZwTGDCQ4Y0SXjrMNHTma0JGjSb23jrhmbkVERNqkljbjkhl8A8u9jiIiIqiw7VOi9XWkGteQXt65T5fNjPCHxhI6YhSpheuIv7xExa2IiPR5zjlSS5sJjBiILxL0Oo6IiKDCtk+J1ueWI8cf6PysrZkRPmYcocNH4hIpUF0rIiJ9XGbDdtzOBAGdu1ZEpGSosO1DAhMHEzhyWJe7I5sZ4ek1RI+fivkMl0xr5lZERPqs1NJm8PsIjh3sdRQREclTYdvHROtrSb64kszaHV3az8xyRW0ixc4/v0mi8X0VtyIi0icFJw4lcux4LBjwOoqIiOSpsO1jovV14CD2p84vR95NKIC/eiDJd1eReH25ilsREelzAlX9CU2p9jqGiIi0o8K2jwnUVRGYMph4F7ojt2dmRI6bQHDycJJvryTx5ooCJxQRESldySUbyDR3bdWTiIh0PxW2fYyZEamvI/Hs+2Q27jz4x5g1keCkYSTfXEFywZoCpxQRESk9Lp0l/tJikgvXeh1FREQ6UGHbB0XrayHjiD+08KAfI1fcTiI0bQyBsUMKmE5ERKQ0pVdthlSGoLohi4iUHBW2fVDw6Gr84wZ2uTtyR+YzIkePxRcN4bJZUss3FSihiIhI6Ukt3YBFg/iHD/Q6iojI/9/evcfXVZX7/v88675yadq06b30fkmBcqsgAgKi0LIRlKiA+BLcW8ELcji6NyoeFC8g8vsdxS3s8xMQ3XjQjVhUVOSigAKygSIItKFQ0tJ7k95Js1ayLs/vj7Ua0vSWtMmaWcn3/XrllTXHmGvOZ4ZJx3rWGHMM6UaJ7RBkZiQb6mn/UxP5bak+OWZH43pSjy6lfcnaPjmeiIjIQOIdWbJrthCdUoeFLOhwRESkGyW2Q1SiYS5k8qR//1qfHC9WP57IlFG0P9tEx1IltyIiA4mZLTCzZWa23My+vI99PmJmS81siZn9vEt5zsxeLP7cX7qoB5bc1jYIh4loGLKIyICkBdiGqNgJEwiNrya1qJGKjx11yMezkJE8dTapvJN+pgnMiNWP74NIRUTkUJhZGLgVeB+wBnjOzO5396Vd9pkJfAU4yd23mtnoLodIufvRJQ16AIqMGUb1hSeAemtFRAYk9dgOURYKkTy/nvSDy8m3tvfdMU+bQ2RSLem/rySfzvTJcUVE5JAcDyx39yZ37wD+Cziv2z6fAm51960A7t5c4hgHNM877o6FQ5gpsRURGYiU2A5hyYZ6SGdp/+PyPjumhUMkT6+n8uyjCCWifXZcERE5aBOA1V221xTLupoFzDKzp8zsv81sQZe6hJktLpZ/oL+DHYgyy9az877n9YWtiMgApsR2CIudMplQXcUhz47cnYVDhEdUAtD+yho6Xt/Yp8cXEZE+FwFmAqcBFwG3m9muqX8nu/t84KPAzWY2vfubzeyyYvK7uKWlpVQxl0ymqQVCpi9sRUQGMCW2Q5iFQyQ+MIf0H17H++FbaM872bVbST/5Gh1vaFSbiEhA1gKTumxPLJZ1tQa4390z7r4CeI1Coou7ry3+bgIeB47pfgJ3v83d57v7/Lq6wTW5Ur41Ta55h9auFREZ4JTYDnHJhrl4awfph9/o82NbyKg4Yy7hsTWkn1hGpknJrYhIAJ4DZprZVDOLARcC3Wc3/g2F3lrMbBSFoclNZjbCzOJdyk8C+naYzwCXaSr0QCuxFREZ2JTYDnHx06dgwxOkFjX2y/EtEqbivYcTHlND6q/LyKwYfEPUREQGMnfPAlcADwGNwC/dfYmZfdPMzi3u9hCw2cyWAo8B/+bum4F6YLGZ/aNYfmPX2ZSHgsyKFsJ11YSqk0GHIiIi+6HlfoY4i0VInDub9P3L8I4sFuv7W8KiheS27ZFX8PZsnx9fRET2z90fAB7oVva1Lq8d+ELxp+s+fwOOLEWMA5G7E583CSLhoEMREZEDUI+tkGyox7elaX9sZb+dw6JhKhbMIzZnHIASXBERGfDMjOjUOqKTaoMORUREDkCJrZA4czpWFevz2ZG7s+Ki9tnmHbz1q+fIrN7cr+cTERE5WO5Ox9K15FvTQYciIiI9oMRWsESUxD/NJP2bV/Fcvt/PF66pIFdJJFsAACAASURBVDQsQerRRrJrtvT7+URERHorv6mV9DNNZNdtCzoUERHpASW2AhRmR863tNHx5Kp+P5fFI1SeeQSh4RW0PbqU7Nqt/X5OERGR3sisKKxdG508KuhQRESkB5TYCgDxhTMgEen34ci7WDxKxVlHEqqpoO3PS8lt2VmS84qIiByI551MUwuRibVYXPNsioiUAyW2AkCoKk5iwQxS9zXi+f4fjgwQShSS29iREwkNryjJOUVERA4kt3E7nurQ2rUiImVEia10SjbUk1/7Fpln15bsnKFElMQxk7GQkd/ZTrZ5R8nOLSIisjf5rTshFiGi2ZBFRMpGSRJbM7vTzJrN7JV91JuZ/buZLTezl8zs2FLEJbtLnDMLoiFSixoDOX/6b8tpe+gVshu3B3J+ERERgNjcCVRfcDym9WtFRMpGqXpsfwos2E/9QmBm8ecy4P+UICbpJjQ8SfyMaaQWLcXdS37+xEkzCVXEaHtkiXpuRUQkEJ4vtH9KakVEyktJElt3/yuwv3VdzgPu8oL/Boab2bhSxCa7SzbUk1uxjcyLG0p+7lBFjIoFR2KJKG0Pv0Ku5a2SxyAiIkNb6olltP25NBMpiohI3xkoz9hOAFZ32V5TLJMSS5w3B0JGukSzI3cXqoxTuWAeFo+SXrwikJ5jEREZmjyTI7tqM5aMBh2KiIj00kBJbHvMzC4zs8VmtrilpSXocAadcF0lsVMnB/acLRRmaK5ceCQV76nHzAKLQ0REhpbs6s2QzROdNjroUEREpJcGSmK7FpjUZXtisWwP7n6bu8939/l1dZqGvz8kG+aSfXUTmaXNgcUQqkpg8Siey5P66zKtcysiIv0u09SCVcQIjxkWdCgiItJLAyWxvR/4eHF25HcC2919fdBBDVXJD84BCLTXdhdPZciu30bbgy+R26rkVkRE+oe3Z8iu3Up0ap1GC4mIlKFSLffzC+BpYLaZrTGzfzGzT5vZp4u7PAA0AcuB24HPliIu2bvw+GHE3jUpsOdsuwpVxalYMA/CIdoefJnctragQxIRkcHIjPg7phKdOSboSERE5CBESnESd7/oAPUOfK4UsUjPJBrq2fHFh8m+sYXI9GAXqA/XJKlYcCRtf3yJtgdfomLhPMI1FYHGJCIig4vFIsTnat5KEZFyNVCGIssAkzy/HoDUAOi1BQjXVBSWAopFIJMPOhwRERlE8m0ddLy2Ac9kgw5FREQOkhJb2avIlBFEjxs3IJ6z3SU8vJLKDxxHeFQVAN6uDyAiInLoMitaSD/1Ovm2jqBDERGRg6TEVvYp2TCXzLNrya7eHnQonSxUmNCj/eU1tP727+TfSgcckYiIlLtMUzOh2ko95iIiUsaU2Mo+JRoKw5HT9w2cXttdIuOH45kcOx98iXyrklsRETk4+R0p8ptatXatiEiZU2Ir+xSdNYrIEaMHzHO2XYVHVlF51pF4R46dD75MvrU96JBERKQMZZpaAIhOrQs4EhERORRKbGW/kg31dDy5ityGt4IOZQ/hUVVUnnkEns7Q9tDLeFaTSomISO/ktu0kPGYYoap40KGIiMghUGIr+5VsmAsO6d+8GnQoexWuq6bizCOIHTUJi+h2FhGR3qk4rZ6K9x0RdBgiInKIlAnIfkWOGE14Zu2Amh25u8joYcRmjAEgu36bZrUUEZEe8bwDYNFwwJGIiMihUmIr+2VmJBvm0v7YCnKb24IOZ788kyX1WCNtD71MPqXkVkRE9s3d2fmb52l/eXXQoYiISB9QYisHlGyoh5yTvn9Z0KHsl0UjJE+vJ/9WmrYHXyafzgQdkoiIDFC55h3kt6ewCj1bKyIyGCixlQOKHjee8OSaATk7cneRccOpeO/cQnL7kJJbERHZu0xTC4RDRA+rDToUERHpA0ps5YDMjMT59bQ/0kR+x8BfMzYyfgQVZ8wlv72NzLL1QYcjIiIDjOfzZFduIjKpFotGgg5HRET6gBJb6ZFkw1zoyJH+/WtBh9IjkQkjqDznGGLzJgUdioiIDDC5ddvwdIboNK1dKyIyWCixlR6JnTiR0LiqAT07cnfh2krMrDAs+bFGvCMbdEgiIjIAhGoqiB01ichEDUMWERkslNhKj1goRPKD9bT/8XXyO8trxuHctjayb26m7eFX8IySWxGRoS5UnSBx7BQsrI9BIiKDhf5Flx5LNtTjqSztDy4POpReiU6qJXnaHHKb3qLtkSV4Jhd0SCIiEpBcy1tkVm/pXMNWREQGByW20mOxd08mNDJZFrMjdxedMorkqXPINe+g7ZFXlNyKiAxR7S+vJv1UecwXISIiPafEVnrMImESH5hD+vev4e3lN6Q3OrWO5LtnQ94hnw86HBERKTHvyJJds4XolDosZEGHIyIifUiJrfRKsmEu/lYH6UfeCDqUgxKdNpqKs4/C4lE8m8ez6rkVERkqMm9uhpwT0WzIIiKDjhJb6ZX4GVOxmjjpMpoduTsLGe5O26NLaXu0Ec+q91ZEZCjINDVjVQnCddVBhyIiIn1Mia30isUiJN4/m9RvXy3r51TNjOiUUeTWbiX12FI8p+RWRAYvM1tgZsvMbLmZfXkf+3zEzJaa2RIz+3mX8kvM7PXizyWli7pveS5PfkeK6LQ6zDQMWURksFFiK72WbKjHt6Zpf3xl0KEcktissSTeNYPsmq2kHmtUcisig5KZhYFbgYXAXOAiM5vbbZ+ZwFeAk9z9cOCqYnkt8HXgBOB44OtmNqKE4fcZC4eo+tA7iM+bFHQoIiLSD5TYSq8lzpqBVUbLcnbk7mKzx5E4cTrZ1VtIP11eyxiJiPTQ8cByd29y9w7gv4Dzuu3zKeBWd98K4O7NxfKzgEfcfUux7hFgQYni7lPujplh0XDQoYiISD9QYiu9ZskoibNnkv71q4OilzM2ZzyJd80gNndC0KGIiPSHCcDqLttrimVdzQJmmdlTZvbfZragF+/FzC4zs8VmtrilpaUPQ+8b+dY0rfc8S3bt1qBDERGRfqLEVg5KomEu+eaddDy1KuhQ+kRs9jjCtZW4O5kVLXjegw5JRKSUIsBM4DTgIuB2Mxve0ze7+23uPt/d59fVDbwZhzNNLXiqg9CwZNChiIhIP1FiKwclcfZMiIdJlfHsyHuT27iD1OOvknpimZJbERks1gJdHyydWCzrag1wv7tn3H0F8BqFRLcn7x3wMitaCNdVE6pOBB2KiIj0EyW2clBC1XESZ80gfV8jni//4ci7RMbWED9uCtmmFtJPvqbkVkQGg+eAmWY21cxiwIXA/d32+Q2F3lrMbBSFoclNwEPAmWY2ojhp1JnFsrKR27aT/JadRLV2rYjIoKbEVg5asqGe3JodZJ5bF3QofSo+bxLxYyaTeaOZ9N9ex13JrYiUL3fPAldQSEgbgV+6+xIz+6aZnVvc7SFgs5ktBR4D/s3dN7v7FuBbFJLj54BvFsvKRqapBQwiU5XYiogMZpGgA5DylXj/bIiESC1aSuyEiUGH06fiRx+Gu9Pxj1VEZ40lMnpY0CGJiBw0d38AeKBb2de6vHbgC8Wf7u+9E7izv2PsL5EJI7BohFAyFnQoIiLSj9RjKwctNCJJ/IyppBY1DspezfjRh1F57rFKakVEylhkTA3xIwfXl68iIrInJbZySJINc8k1bSXzjw1Bh9LnzIxwbSUAmdWbST/bNCgTeBGRwSqzajO5La1BhyEiIiWgxFYOSeIDcyBkpAfZ7Mjd5TbuoGPJWtqV3IqIlAXPO+m/vU77C4NjWToREdk/JbZySMJ1lcTePZnUoqVBh9Kv4sdNITZ3PB1L19H+3AoltyIiA1xuw3Y8ldFsyCIiQ4QSWzlkyYZ6so2byDS2BB1KvzEz4sdPI1o/jo4la2n740uddZ4dPMsdiYgMFpkVLRAJE5lUG3QoIiJSAiVLbM1sgZktM7PlZvblvdRfamYtZvZi8eeTpYpNDk3yg/UAg77X1sxInDCdxAnTCY+t6Szf+bsXaP3186SeXk5mZQv5VEeAUYqIiOfyZFZuIjp5JBYJBx2OiIiUQEmW+zGzMHAr8D5gDfCcmd3v7t0zoXvc/YpSxCR9JzxhGLETJ5Je1Miw/3Vq0OH0KzMjNnd857a7E50+muz6bWSWbyTz6noAYodPIHH8NNwdT2e0zISISAnlt7ZBPq+1a0VEhpBSrWN7PLDc3ZsAzOy/gPOAwd3FN4QkGuay418fJtu0hci0oTPsy8yIz5tEfN4kPJ8nt6mV3IbthIqzKXtrO62/eo5QTZLw2BoiY2sIjx1OqEKJrohIfwmPqqL6wndCWE9ciYgMFaX6F38CsLrL9ppiWXcNZvaSmf3KzCaVJjTpC8nzdw1HHtyzI++PhUJERg8jPm8S0YnF5D4SIj5/CqHqBJmmFlJ/WUbrPc+QWbUZgHw6Q35ne4BRi4gMLrsm97NoGAtZwNGIiEiplKrHtid+B/zC3dvN7HLgP4H3dN/JzC4DLgM47LDDShuh7FNk6giix44jtWgp1f92UtDhDBihZIz4kZPgyEl43slvaSW7fjvhumoAMm800/5sE6HqBOGxNYTHDScypoZQVTzgyEVEylN2RQvtL62h4szDCVXo31IRkaGiVD22a4GuPbATi2Wd3H2zu+/quroDOG5vB3L329x9vrvPr6vTszMDSfL8ejLPrCW3ZnvQoQxIFjLCo6qJHzmx85nbyMRa4sdPIzS8gsybm0n/dRmtv3oWz+QAyG3ZSb41HWTYIiJlJdPUgrdnMM1tICIypJSqx/Y5YKaZTaWQ0F4IfLTrDmY2zt3XFzfPBYbumNYylWioZ8f/epTUr1+l6vMnBB1OWQjXJAnXTIDDJ+Du5LfuJL+1DYsWZvFMP/MGuQ3bsap45/O5kbE1hKoTAUcuIjLweHuG7NqtxOaOx0zDkEVEhpKS9Ni6exa4AniIQsL6S3dfYmbfNLNzi7tdaWZLzOwfwJXApaWITfpOdE4dkbl1g37Zn/5iZoRrq4hOH91ZlnjndBInTCM8sors6i2kn3yN1N9e76zPrNxEfkeq85kyEZGhLLNyE+Sd6NTRB95ZREQGlZI9Y+vuDwAPdCv7WpfXXwG+Uqp4pH8kG+p56/onyDW3Eh5dFXQ4ZS88opLwiEpic4s9utvaIJcHwDuypB5vBAeriHXOuhyZUKtndEVkSMo0tRCqSRIaWRl0KCIiUmKaB1/6VLJhLuSd9G9eDTqUQcfMConuqMLEU0TDVH7gOBInTic8Zhi59dtI/2052VWbAMinOuh4dT25bW3q0RWRISE2ayzxow/TMGQRkSFoIM2KLINAZN4YwtNHkFrUSOVl84MOZ1AzM8LDKwgPryA2Z3yhR3dHCosX/rfObdhO+unlhX2TUcJjaoiMqyE6tQ6LR4MMXUSkX3R9lENERIYW9dhKnzIzkg1zaX90BfmtqaDDGVLMjHBNBaFEccblKaOoPH8+iXfNIDJuOLnmHaSffqNzxuXs2q10NK4jt3WnenRFpOx1LN9Ivk3rgouIDFXqsZU+l2yop/Wmp0jdv4zKS44OOpwhq5DoJgnXJGH2ONwdb20nVFWYUTnz5iYyyzYU9o1His/oDidaP07D+ESkrOS2p0g/8Rrxd0wlfsTEoMMREZEAqMdW+lz0HRMITxpGWrMjDyhmttsyQYkTZ1D1oXeQOHkmkUm15Da30vHahs6ktv3FVbQvWUtuc6t6dEVkQMuuaAYgOlXr24uIDFXqsZU+Z2Ykzq9n5/+3mPxb7YSqNUPvQGRmWHWCWPVYmDkWKMy0DODuZN7cRH7LTtoBYhEiY4YRnTGa6BR9cBSRgcPdyTS1EB4zjFCl2hsRkaFKPbbSL5INc6E9R/oPrwUdivSCxQrfdZkZVecdS9WH30HilFlEJ48kt62N3JadAHgmR9ufltD+yhpym97C8+rRFZFg5LfsJL89RXSaJo4SERnK1GMr/SL2rkmExlSSWtRIxYVHBh2OHKRQVYLYjATMGAPQmcDmd7aT354iu3pLoUc3GiYyehjx+VMJ11aS35Eis3oLFg1j0TBECr/DtZVYLIJn8+AOkZCe5xWRQ5JreQvCRmTKqKBDERGRACmxlX5h4RDJD9bTdtc/yLd1EKqIBR2S9AELFZLQ8PAKqhrmk29rJ7dhO9kN28lt3NE5lDm3uZX2Z5v2eH/FPx1FZPQwMiuaST/5eqEwEiokwJEwyfcdTrimgszqzWSWN3eWU0yQY3PGYbEIue1teGt78b0RLBoqJM+JqBJlkSEmNmdccRkzfaQRERnK1ApIv0k0FJ6zbX/oDZIfrA86HOkHoYo4oWmj9xgCGJk8iuqPvhPP5PBsrrDEUCZHeHgFAOGR1cTnT8WzhfJd+1m08E+Sp7Pkt+7sLCeTA4fozDEYkGlqoePFVXvEU/2xEyEaIf33lWRe37hbb7FFwyTPmIuZkVm5idzWnVgk/HZSHIsQPWwkUOiRxv3t94f11IbIQOTuhfkClNSKiAx5agmk38RPnYLVJkktWqrEdoixkEE8isWje60P11YSrq3c5/tjM8cQmzmmc9vdIZeHYoIZmz2WyPgRUEyaPZODbA4i4cLxR1TiE0Z0lnsmh6cznb252TVbyLy+cfeY4xGiHz0RgPQzb5B9c/PblSEjVJOk6gPHddbntrZ1GWodIlSdJH5kYZmRzKrNnfFYNAwhw6KRzmvOtbyF5/J7nD88olCf3bgD8t3qk1HCw4v1G7ZBHuDtZ5utIt75xUF23dbiH67L+6vihGsq8LyTW79tj795qDpBaFgSz+YLx+8mXFNBqDqBZ3JkN2zfs35EBaGqBN6eJbtxL/UjqwhVxsmnM+Q27ijEFDaIRrBomFB1AouGOxMVkZ5of6aJ/M52ku+p130jIjLEKbGVfmPRMMnzZpNa1Ii3Z/WNuhw0M+tMWqHYU1yx79lPo1Pr9rvsR/LkWSROmgnZ/NvJb5dEMzZ3ApGJtbv1KFuX82MGuTz5dKZzn1DN24lt+wtvki9OtLVLeGwNlQvnAZD66zLyO1K71Ucm1VLx3sML9Y8txVOZ3eun1VFx6hwA2h5ZAtndE9/orLEkT5pZqH/olT2uOXb4BMLHT4NcjraH91J/9GEkjpmMd2RIPbJkj/pd64Pm29pJ/WnP+sS7ZhCbPY78WylSf95zqa/kqbMJTRtNfutOUo/upf69c4lOGkl29RZSj79a+EIg+nZve+Kd0wnXVpHb9BaZphYsFu5MisMjKgnXVe9xTBncPJ8n09RMePxwJbUiIqLEVvpXsmEubT95kfY/N5E4e1bQ4Yh0MrPOxKm7yNgaGFuzz/cmjp+232NXvO+IwvPGu4Zhu3fOOA2QOGXWHompJd6uT55eD91mmrbk273fFWcesVtvbKH+7efYKxbOg26f8zu/CAiHqTj7qC5v3L3e4lEqzjmK7kKVieLvOJXnHL1HvVUV3h+qqaDy/V3rCyfYtexXeFQVleceA15ITApfHGQJj6wu7pcgVj8ez2Q7h7B7Jlf4MgHIbU/RsWz9bn+/WP14JbZDUG7dNrw9S3SqZkMWERElttLP4u+dhg2Lk1rUqMRWhoxQRQz2M2FaZPSw/b4/MmbfSXWP6veTlFvIiIzZ9/ktHCJSt5/6SHi/SaRFw4RH7a8+Qnhk1T7rwyMqCb9j6j7rY9NHE5s+ujBDdzEpRs9AD0mZppbCGtsTRwQdioiIDAD6NCD9yuIREufMIv3bVwtDNkVE+oCFChMGhaoShJKadX2o8WyOzJubiU4eqcndREQEUGIrJZBsqCe/OUX7X94MOhQRERkM3IkffRjR2WODjkRERAYIJbbS7+ILZmAVUdKL9pwwRkREpLcsGiF+5MT9DpsXEZGhRYmt9LtQRYz4whmkfv1qYbIYERGRg+QdWTJNzYVJxURERIqU2EpJJBvmkt/QSsffVgcdioiIlLHMm5tJ/WUZuW07D7yziIgMGUpspSQS/zQTYmFSixqDDkVERMpYpqkZq07sd/ZtEREZerTcj5REaFiCxJnTSd/XiH/vrMIaoiIiIr2QT3WQW7+N2JGT1I6IBGjHjh00NzeTyWSCDkXKXDQaZfTo0QwbduhzJiixlZJJNNST/v1rZBavI/aOCUGHIyIyZJjZAuAHQBi4w91v7FZ/KfD/AGuLRbe4+x3FuhzwcrF8lbufW5Kg9yK7YhM4RKfVBRWCyJC3Y8cONm7cyIQJE0gmk/qSSQ6au5NKpVi7ttD0HGpyq8RWSiZ57my2RUKkFi1VYisiUiJmFgZuBd4HrAGeM7P73b37VPX3uPsVezlEyt2P7u84eyLXsoPQiArCIyqDDkVkyGpubmbChAlUVFQEHYqUOTOjoqKCCRMmsG7dukNObPWMrZRMqLaC+OlTSC1qxN2DDkdEZKg4Hlju7k3u3gH8F3BewDEdlMS7Z1O5cF7QYYgMaZlMhmQyGXQYMogkk8k+GdauxFZKKtkwl9zyLWRf3hh0KCIiQ8UEoOuU9GuKZd01mNlLZvYrM5vUpTxhZovN7L/N7AP9Gul+uDtmhsWjQYUgIkUafix9qa/uJyW2UlKJD8wBQ7Mji4gMLL8Dprj7POAR4D+71E129/nAR4GbzWx69zeb2WXF5HdxS0tLvwTY9seXaH9JS8aJiMjeKbGVkgqPqSJ2ymRSi7o/2iUiIv1kLdC1B3Yib08SBYC7b3b39uLmHcBxXerWFn83AY8Dx3Q/gbvf5u7z3X1+XV3fT+yU27qT3MYdWEQfW0REZO/UQkjJJRvqyS5pIbNsU9ChiIgMBc8BM81sqpnFgAuB+7vuYGbjumyeCzQWy0eYWbz4ehRwElDybyYzTS1gEJmq2ZBF5OBdeumlhUcazIhGo4waNYqTTz6Zm266iZ07dwYdnhwiJbZScsnz6wFI/ewf5Lem8Fw+4IhERAYvd88CVwAPUUhYf+nuS8zsm2a2a+meK81siZn9A7gSuLRYXg8sLpY/Bty4l9mU+zt+MitaCI8bTigZK+WpRWQQOuWUU1i/fj1vvvkmjz32GBdffDG33HILxx57LBs3Dt05YDo6OoIO4ZApsZWSC0+sIfrOibx1/ROsr/0u6yLfZF31Dayf8L/ZOPcWmk+8g01n/YwtH/klWz91P9u/+BA7vvk4rT/4b3b+5AVS9y0l/ecmOp5bS+a1TeQ2tuKpjGZaFhHZB3d/wN1nuft0d7++WPY1d7+/+Por7n64ux/l7qe7+6vF8r+5+5HF8iPd/celjj2/qRV/K621a0WkT8RiMcaOHcv48eM58sgj+cxnPsPTTz9NS0sLX/7yl3fb94c//CFz5swhkUgwc+ZMrr/+erLZLABf/epXmT179h7H/8xnPsPJJ5+8z/M/8sgjnHbaadTW1lJTU8Opp57Ks88+u9s+ra2tXHXVVUyaNIl4PM6UKVO44YYbOuubm5v5xCc+wZgxY0gkEsyePZs777wTgMcffxwzY82aNbsdMxKJ8NOf/hSAlStXYmbcfffdnH322VRWVnLttdfi7nzqU59i+vTpJJNJpk2bxjXXXEN7e/tux/rTn/7EKaecQkVFRec1vPHGGzz++OOEw2FWr959PoS77rqLmpqafu8V1zq2Eojau8+n/fGV+PZ28jvayW9Pd7727Wny29LkVm3vLPe2HkwBHg0Rqklgw+KEauJYTYLQsDhWE9+tfL/7VMewkL7vEREZKCwRIXb4BKKHjQo6FBEZpCZMmMDFF1/MXXfdxY9//GNCoRDXXXcdP/nJT7j55ps5+uijaWxs5NOf/jTpdJpvfetbXHLJJdxwww0888wznHDCCQC0t7dzzz33cOONN+7zXK2trXz2s5/lqKOOIpvN8v3vf58FCxbw+uuvM3LkSNydc845h1WrVvHDH/6QefPmsWbNGpYtWwZAKpXi1FNPJZlMcvfddzNt2jSWL1/Oli1ben3dX/rSl/jud7/LrbfeChRGyIwePZqf//znjBkzhpdeeonLL7+caDTKN77xDaCQ1J511ll8/vOf55ZbbiEej/PUU0+RyWQ47bTTmDlzJnfeeSdf//rXO89z++2389GPfpTKyv5dg1yJrQQiMq2WyLTaHu/v2Ry+o5389vbi7/Rur3dLiruUZ1du220f8gfu1bXqWI8S5F3le90nrv+1RET6Qqg6SeL4aUGHISIHsPOPL+1RFp0yilj9eDybo+2RJXvWzxhDbOYY8ukMqcf2XDEjNnsc0Wl15FvbST2xbI/6vlzX+vDDD2fHjh1s2rSJqqoqbrrpJu677z4WLFgAwNSpU/n2t7/NlVdeybe+9S1mzZrFCSecwF133dWZ2P7ud78jlUrxkY98ZJ/n+eAHP7jb9m233caiRYt48MEHufjii3n00Uf5y1/+wnPPPcf8+fMBmDZtGu9+97sB+PnPf86KFStYvnw5EydO7Kw/GJdffjkXX3zxbmXXX3995+spU6bwxhtv8B//8R+die03vvENFi5cyM0339y535w5czpfX3bZZfzgBz/g2muvJRQK8eqrr/Lkk0/y7//+7wcVY2/o07eUBYuEsdoKQrUVB30Md8fbMnskv/vsNd61z+Y2ck1bO8s9lT3wyWLhg+813lVepd5jERnactt24qkM4TE1WEjrZopI/9n1SJuZsWTJElKpFA0NDbutsZrL5Uin07S0tFBXV8cll1zCtddey80330w0GuWuu+7i3HPPZfjw4fs8z4oVK/ja177G008/TXNzM/l8nra2Nt58800Ann/+eUaMGNGZ1Hb3/PPPM3fu3M6k9lAcf/zxe5Tdfvvt3HHHHaxcuZKdO3eSzWbJ59+eD+f555/fb4/0JZdcwle/+lUeeughFi5cyB133MFxxx3HMcfsMaF+n1NiK0OGmWGVMaiMER5/8Mfxjiz5tzr2mSB3L89vL5Rl39hS2Gd7Gt/RDgfqPDaw6kMYVr2rPKb/zUWkPHUsWUemqYXqi06AUDjocERkP/bXe2qRsfT6ygAAEDdJREFU8H7rQ4no/uur4n3aO7s3S5YsoaamhpEjR9LU1ATAvffey6xZs/bYt7a2MOrwwgsv5KqrruIPf/gDJ510Eg8++CC/+c1v9nuec845h1GjRnHrrbcyadIkYrEYJ598cp9N3hQqdop0nXsml8vtlpzu0n1o8L333svnPvc5brzxRk499VSGDRvGvffey1e/+tUen3/kyJF86EMf4vbbb+eMM87grrvu4tvf/vZBXk3v6BOvSC9ZLEJ4ZARGHmLvcWsxOd41xLr4umvyu0d5806yy99OkGnPHfhkicjbCe+wOJaMYolI4ScZgV2vE5E96izRpb6HdURCu327KSJyMDyXJ7NyE9HJI7GIkloR6T9r167l7rvv5vzzzycUCnH44YeTSCRoamri7LPP3uf7RowYwfvf/35+9rOfsWrVKmpraznrrLP2uf/mzZtZunQpDzzwQOd+a9asobm5uXOf4447jq1bt7J48eK99toed9xx3HnnnaxZs2avvbajR48GYN26dUyaVFjC/MUXX+zRJKt//etfOeaYY/jCF77QWbZy5co9zv/www9z5ZVX7vM4l19+Oaeffjo/+tGPSKVSXHTRRQc8d18oWWJrZguAHwBh4A53v7FbfRy4i8Ki8JuBC9x9ZaniEyklM8Oq41Ad51A+rnl79u1e4r0lxd2fR97RXphBemcH+c1teDqLp7KF3+ksnsr0LFnen5Dtnvh2TYi7JMV7re9SZ4kI7Oe9lox2S8oj+vArMohk126FjiwRzYYsIn2oo6ODDRs2kM/n2bx5M08++STf+c53GD16NN/5zncAqKqq4pprruGaa67BzHjve99LNpvl5Zdf5oUXXuC73/1u5/E+/vGP8+EPf5jGxkYuvvhiwuF9fxYZMWIEdXV13H777UyfPp3Nmzdz9dVXk0wmO/d5z3vewymnnMIFF1zA9773PebNm8e6detobGzkk5/8JBdddBE33XQT5557LjfddBPTp0+nqamJTZs2ccEFFzBjxgwmT57Mddddx/e//302bdrUeR0HMnv2bH784x/z29/+liOOOILf//733Hfffbvtc+2117Jw4UKuuuoq/vmf/5l4PM7TTz/NiSee2DlL9Mknn8zs2bP513/9Vz7+8Y9TXV3dq/9GB6skia2ZhYFbgfcBa4DnzOz+bmvh/Quw1d1nmNmFwHeBC0oRn0i5sniEcF0E6vpuljnP56Ejt9ek19NZ6NzuUte9fh91ns7irR3kW9r2WnfISXXYdutBZh8J825J8V7q9vnerj3T3euUVIv0qUxTCxaPEBm/72fVRER664knnmDcuHGEw2Fqamqor6/niiuu4HOf+9xuQ3OvvfZaxo0bxy233MIXv/hFkskks2bN4tJLL93teAsXLqSmpobGxkZ+8Ytf7PfcoVCIe++9lyuvvJJ58+YxefJkbrjhBr70pS917mNm/OEPf+Caa67h05/+NJs3b2bChAlcfvnlAFRUVPCXv/yFq6++mgsvvJDW1lamTJnSuVRRJBLhnnvu4bOf/SzHHHMMs2bN4pZbbuH0008/4N/m8ssv5+WXX+YTn/gE2WyWc845h+uuu47Pf/7znfuceeaZPPDAA1x33XX86Ec/IhaLceyxx3ZObrXLpz71Ka666iouu+yyA563r1gp1v40sxOB69z9rOL2VwDc/Ttd9nmouM/TZhYBNgB1vp8A58+f74sXL+7f4EWkZDyfh/bcvpPiVJfEuntd9/q91O2emHc7bkcfJNVdE98uSW/yQ3Opvnrfa9rJwGFmz7v73mfskB7pi7bZ807roueITKwleeKMPopMRPpCY2Mj9fX1QYchA9zVV1/NI488wgsvvNCj/fd3X/W0bS7VUOQJQNeVetcAJ+xrH3fPmtl2YCSwqSQRikjgLBSCZKjQo1piuyXV+0uK91G31/pinSU0nYFIb1jIqGp4B2R6MAu9iIgMGNu3b+e1117jtttuK8kSP12V3actM7sMuAzgsMMOCzgaERksdkuqRyQP/AYR6VcWMoiX/ksuERE5eOeddx7PPPMMF154IR/72MdKeu5SJbZrgUldticWy/a2z5riUOQaCpNI7cbdbwNug8Jwp36JVkRERERERHrl8ccfD+zcoRKd5zlgpplNNbMYcCFwf7d97gcuKb7+EPDo/p6vFREREREREYES9dgWn5m9AniIwnI/d7r7EjP7JrDY3e8Hfgz8zMyWA1soJL8iIiIiIiIi+1WyZ2zd/QHggW5lX+vyOg18uFTxiIiIiIhI77l7j9ZFFemJvhqkW6qhyCIiIiIiUuai0SipVCroMGQQSaVSRKOHPlmgElsREREREemR0aNHs3btWtra2vqsp02GJnenra2NtWvXMnr06EM+Xtkt9yMiIiIiIsEYNmwYAOvWrSOTyQQcjZS7aDTKmDFjOu+rQ6HEVkREREREemzYsGF9koiI9CUNRRYREREREZGypsRWREREREREypoSWxERERERESlrSmxFRERERESkrCmxFRERERERkbJm5bz+lJm1AG/20eFGAZv66FilVK5xg2IPQrnGDYo9KOUa+8HGPdnd6/o6mKGkj9vmUivX+723dJ2Dx1C4RtB1DiYHc409apvLOrHtS2a22N3nBx1Hb5Vr3KDYg1CucYNiD0q5xl6ucUuwhsp9o+scPIbCNYKuczDpz2vUUGQREREREREpa0psRUREREREpKwpsX3bbUEHcJDKNW5Q7EEo17hBsQelXGMv17glWEPlvtF1Dh5D4RpB1zmY9Ns16hlbERERERERKWvqsRUREREREZGyNuQTWzNbYGbLzGy5mX056Hj2x8zuNLNmM3ulS1mtmT1iZq8Xf48IMsa9MbNJZvaYmS01syVm9j+K5eUQe8LMnjWzfxRj/0axfKqZPVO8b+4xs1jQse6LmYXN7AUz+31xuyxiN7OVZvaymb1oZouLZeVwzww3s1+Z2atm1mhmJ5ZJ3LOLf+tdPzvM7KpyiB3AzP5n8f/RV8zsF8X/d8viXpdglHPb1BuDoR3rqXJt73qjXNvG3ijXdrQ3yr3N7Y1Sts9DOrE1szBwK7AQmAtcZGZzg41qv34KLOhW9mXgz+4+E/hzcXugyQJfdPe5wDuBzxX/zuUQezvwHnc/CjgaWGBm7wS+C3zf3WcAW4F/CTDGA/kfQGOX7XKK/XR3P7rLtPDlcM/8AHjQ3ecAR1H42w/4uN19WfFvfTRwHNAG/JoyiN3MJgBXAvPd/QggDFxIed3rUnrl3Db1xmBox3qqnNu73ijHtrE3yrId7Y1ybnN7o+Tts7sP2R/gROChLttfAb4SdFwHiHkK8EqX7WXAuOLrccCyoGPswTX8FnhfucUOVAB/B06gsLB0ZG/30UD6ASZS+IfxPcDvASuj2FcCo7qVDeh7BqgBVlCcv6Bc4t7LdZwJPFUusQMTgNVALRAp3utnlcu9rp+B8VOubVMvr7Hs2rFeXFvZtne9vM6yaxt7eX2Doh3t5TWXVZvby2srafs8pHtsefuPvcuaYlk5GePu64uvNwBjggzmQMxsCnAM8AxlEntxaNOLQDPwCPAGsM3ds8VdBvJ9czNwNZAvbo+kfGJ34GEze97MLiuWDfR7ZirQAvykOBzuDjOrZODH3d2FwC+Krwd87O6+Fvh/gVXAemA78Dzlc69LwMqxbeqNMm/Heqqc27veKMe2sTcGSzvaG2XV5vZGqdvnoZ7YDipe+NpjwE5zbWZVwCLgKnff0bVuIMfu7jkvDBWZCBwPzAk4pB4xs3OAZnd/PuhYDtLJ7n4shUcFPmdm7+5aOUDvmQhwLPB/3P0YYCfdhhEN0Lg7FZ9zORe4t3vdQI29+AzSeRQ+EI0HKtnzsQ2RvSrXtqk3yrUd66lB0N71Rjm2jb1R9u1ob5Rjm9sbpW6fh3piuxaY1GV7YrGsnGw0s3EAxd/NAcezV2YWpfDB4W53v69YXBax7+Lu24DHKAyZGG5mkWLVQL1vTgLONbOVwH9RGJ71A8oj9l3f8uHuzRSeOzmegX/PrAHWuPszxe1fUWigB3rcXS0E/u7uG4vb5RD7e4EV7t7i7hngPgr3f1nc6xKcwdA29UYZtmM9VdbtXW+UadvYG4OhHe2Ncmxze6Ok7fNQT2yfA2YWZ+aKURgKcH/AMfXW/cAlxdeXUHhGaEAxMwN+DDS6+/e6VJVD7HVmNrz4Oknh+atGCh8MPlTcbUDG7u5fcfeJ7j6Fwr39qLtfTBnEbmaVZla96zWF509eYYDfM+6+AVhtZrOLRWcASxngcXdzEW8PiYLyiH0V8E4zqyj+e7Pr7z7g73UJTjm3Tb1Rzu1YT5Vze9cb5do29sYgaUd7oxzb3N4oaftsxYd2hywzO5vCcxlh4E53vz7gkPbJzH4BnAaMAjYCXwd+A/wSOAx4E/iIu28JKsa9MbOTgSeAl3n72ZdrKDzLNNBjnwf8J4X7IwT80t2/aWbTKHwrXAu8AHzM3duDi3T/zOw04F/d/ZxyiL0Y46+LmxHg5+5+vZmNZODfM0cDdwAxoAn4BMV7hwEcN3R+UFoFTHP37cWyAf83B7DCEiYXUJjp9gXgkxSe2RnQ97oEp5zbpt4YLO1YT5Vbe9cb5dw29kY5t6O9Uc5tbm+Usn0e8omtiIiIiIiIlLehPhRZREREREREypwSWxERERERESlrSmxFRERERESkrCmxFRERERERkbKmxFZERERERETKmhJbERERERERKWtKbEUGETO7zsz+b9BxiIiISIHaZpHSUGIrIiIiIiIiZU2JrUiZMrMvmdlaM3vLzJaZ2T8B1wAXmFmrmf2juF+Nmf3YzNYX9/+2mYWLdZea2VNmdouZbTezV83sjCCvS0REpFypbRYJTiToAESk98xsNnAF8A53X2dmU4AwcAMww90/1mX3nwLNwAygEvg9sBr4UbH+BOBXwCjgfOA+M5vq7lv6/0pEREQGB7XNIsFSj61IecoBcWCumUXdfaW7v9F9JzMbA5wNXOXuO929Gfg+cGGX3ZqBm9094+73AMuAf+r/SxARERlU1DaLBEg9tiJlyN2Xm9lVwHXA4Wb2EPCFvew6GYgC681sV1mIwrfCu6x1d++y/SYwvs+DFhERGcTUNosESz22ImXK3X/u7idTaCAd+G7xd1ergXZglLsPL/4Mc/fDu+wzwbq0rMBhwLr+jF1ERGQwUtssEhwltiJlyMxmm9l7zCwOpIEUkAc2AlPMLATg7uuBh4H/bWbDzCxkZtPN7NQuhxsNXGlmUTP7MFAPPFDSCxIRESlzaptFgqXEVqQ8xYEbgU3ABgoN4FeAe4v1m83s78XXHwdiwFJgK4XJKMZ1OdYzwMzisa4HPuTum/v7AkRERAYZtc0iAbLdh++LyFBiZpcCnywOmxIREZGAqW0WOTjqsRUREREREZGypsRWREREREREypqGIouIiIiIiEhZU4+tiIiIiIiIlDUltiIiIiIiIlLWlNiKiIiIiIhIWVNiKyIiIiIiImVNia2IiIiIiIiUNSW2IiIiIiIiUtb+fwDV/cbDIi/wAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 学习率大小\n",
    "lr = 0.01  \n",
    "# 批次大小\n",
    "batch_size = 64\n",
    "# 加载数据\n",
    "train_loader = io.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)\n",
    "dev_loader = io.DataLoader(dev_dataset, batch_size=batch_size)\n",
    "test_loader = io.DataLoader(test_dataset, batch_size=batch_size)\n",
    "# 定义网络，通过指定use_residual为True，使用残差结构的深层网络\n",
    "model = Model_ResNet18(in_channels=1, num_classes=10, use_residual=True)\n",
    "# 定义优化器\n",
    "optimizer = opt.SGD(learning_rate=lr, parameters=model.parameters())\n",
    "# 实例化RunnerV3\n",
    "runner = RunnerV3(model, optimizer, loss_fn, metric)\n",
    "# 启动训练\n",
    "log_steps = 15\n",
    "eval_steps = 15\n",
    "runner.train(train_loader, dev_loader, num_epochs=5, log_steps=log_steps, \n",
    "            eval_steps=eval_steps, save_path=\"best_model.pdparams\")\n",
    "\n",
    "# 可视化观察训练集与验证集的Loss变化情况\n",
    "plot_training_loss_acc(runner, 'cnn-loss3.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 5.4.3.2 模型评价\n",
    "\n",
    "使用测试数据对在训练过程中保存的最佳模型进行评价，观察模型在测试集上的准确率以及损失情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Test] accuracy/loss: 0.8800/0.3523\n"
     ]
    }
   ],
   "source": [
    "# 加载最优模型\n",
    "runner.load_model('best_model.pdparams')\n",
    "# 模型评价\n",
    "score, loss = runner.evaluate(test_loader)\n",
    "print(\"[Test] accuracy/loss: {:.4f}/{:.4f}\".format(score, loss))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "添加了残差连接后，模型收敛曲线更平滑。\n",
    "从输出结果看，和不使用残差连接的ResNet相比，添加了残差连接后，模型效果有了一定的提升。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.4.4 与高层API实现版本的对比实验\n",
    "\n",
    "对于Reset18这种比较经典的图像分类网络，飞桨高层API中都为大家提供了实现好的版本，大家可以不再从头开始实现。这里为高层API版本的resnet18模型和自定义的resnet18模型赋予相同的权重，并使用相同的输入数据，观察输出结果是否一致。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.0.0.bias. net.0.0.bias is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.2.0.bn3.weight. net.2.0.bn3.weight is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.2.0.bn3.bias. net.2.0.bn3.bias is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.2.0.bn3._mean. net.2.0.bn3._mean is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.2.0.bn3._variance. net.2.0.bn3._variance is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.3.0.bn3.weight. net.3.0.bn3.weight is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.3.0.bn3.bias. net.3.0.bn3.bias is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.3.0.bn3._mean. net.3.0.bn3._mean is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.3.0.bn3._variance. net.3.0.bn3._variance is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.4.0.bn3.weight. net.4.0.bn3.weight is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.4.0.bn3.bias. net.4.0.bn3.bias is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.4.0.bn3._mean. net.4.0.bn3._mean is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py:1441: UserWarning: Skip loading for net.4.0.bn3._variance. net.4.0.bn3._variance is not found in the provided dict.\n",
      "  warnings.warn((\"Skip loading for {}. \".format(key) + str(err)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,\n",
      "       [0.])\n"
     ]
    }
   ],
   "source": [
    "from paddle.vision.models import resnet18\n",
    "import warnings\n",
    "#warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "# 使用飞桨HAPI中实现的resnet18模型，该模型默认输入通道数为3，输出类别数1000\n",
    "hapi_model = resnet18()\n",
    "# 自定义的resnet18模型\n",
    "model = Model_ResNet18(in_channels=3, num_classes=1000, use_residual=True)\n",
    "\n",
    "# 获取网络的权重\n",
    "params = hapi_model.state_dict()\n",
    "# 用来保存参数名映射后的网络权重\n",
    "new_params = {}\n",
    "# 将参数名进行映射\n",
    "for key in params:\n",
    "    if 'layer' in key:\n",
    "        if 'downsample.0' in key:\n",
    "            new_params['net.' + key[5:8] + '.shortcut' + key[-7:]] = params[key]\n",
    "        elif 'downsample.1' in key:\n",
    "            new_params['net.' + key[5:8] + '.shorcutt' + key[23:]] = params[key]\n",
    "        else:\n",
    "            new_params['net.' + key[5:]] = params[key]\n",
    "    elif 'conv1.weight' == key:\n",
    "        new_params['net.0.0.weight'] = params[key]\n",
    "    elif 'bn1' in key:\n",
    "        new_params['net.0.1' + key[3:]] = params[key]\n",
    "    elif 'fc' in key:\n",
    "        new_params['net.7' + key[2:]] = params[key]\n",
    "\n",
    "# 将飞桨HAPI中实现的resnet18模型的权重参数赋予自定义的resnet18模型，保持两者一致\n",
    "model.set_state_dict(new_params)\n",
    "\n",
    "# 这里用np.random创建一个随机数组作为测试数据\n",
    "inputs = np.random.randn(*[1,3,32,32])\n",
    "inputs = inputs.astype('float32')\n",
    "x = paddle.to_tensor(inputs)\n",
    "\n",
    "output = model(x)\n",
    "hapi_out = hapi_model(x)\n",
    "\n",
    "# 计算两个模型输出的差异\n",
    "diff = output - hapi_out\n",
    "# 取差异最大的值\n",
    "max_diff = paddle.max(diff)\n",
    "print(max_diff)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "可以看到，高层API版本的resnet18模型和自定义的resnet18模型输出结果是一致的，也就说明两个模型的实现完全一样。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
